Method and device for controlling reactive voltage of power distribution network in measurement missing environment

Abstract

The invention discloses a power distribution network reactive voltage control method in a measurement missing environment, relates to the technical field of power distribution network voltage control, and constructs a power distribution network environment with large measurement range missing. Establishing a DAE measurement completion model of a noise reduction auto-encoder based on a GCN (Graph Convolutional Neural Network), including the steps of mining spatial relevance of each finite physical quantity by a GCN encoder, and restoring voltage information of a measurement missing node by a DNN decoder; establishing a dual-time scale voltage control model in the measurement missing environment, wherein control equipment comprises a slow time scale OLTC and a fast time scale PV inverter; multi-agent completely observable Markov process modeling is performed on a control process, and an improved MASAC algorithm is adopted to train a dual-time scale agent. The invention further discloses a reactive voltage control device for the power distribution network in the measurement missing environment, model-free voltage control under measurement large-range missing is realized, the deployment cost of measurement equipment is reduced, and the timeliness of voltage control is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of distribution network voltage control, and in particular to a distribution network reactive voltage control method and device under a measurement-missing environment. Background Art

[0002] The widespread integration of distributed renewable energy not only promotes the low-carbon transformation of distribution networks, but also provides a new source of reactive power compensation for advanced power electronic devices. However, due to the variable output characteristics of distributed renewable energy, state perception and voltage control in distribution networks face huge challenges. The output of distributed renewable energy is characterized by randomness and volatility, which will rapidly change the voltage state of the distribution network. Therefore, the control center not only needs to obtain global accurate voltage measurements, but also needs to adopt more advanced control algorithms, such as deep learning and deep reinforcement learning to replace traditional algorithms. These algorithms have uncertainty data mining capabilities and fast decision-making speeds to cope with the uncertainty of distribution network sources and loads. In terms of system voltage monitoring, due to the limitations of data transmission channels and the challenges of deploying full-node measurement facilities, the control center can only obtain real-time measurement information from some nodes during the actual control process, resulting in the control agent's lack of perception of the global state. Summary of the invention

[0003] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a method and device for reactive voltage control of a distribution network in a measurement-missing environment. Based on deep learning and deep reinforcement learning techniques, measurement completion agents and multi-time-scale voltage control agents are trained, and the trained models are put into practical application.

[0004] The present invention adopts the following technical solutions to solve the above technical problems:

[0005] A reactive voltage control method for a distribution network in a measurement-missing environment according to the present invention includes:

[0006] Construct a distribution network environment with large-scale missing measurements;

[0007] A denoising autoencoder DAE measurement completion model based on graph convolutional neural network GCN is established, wherein the denoising autoencoder measurement completion model includes a GCN encoder and a DNN decoder, wherein the GCN encoder is used to mine the spatial correlation of each finite physical quantity, and the DNN decoder is used to restore the voltage information of the missing measurement node;

[0008] Establishing a dual-time-scale voltage control model in a distribution network environment with large-scale measurement loss, wherein the control devices in the dual-time-scale voltage control model include a slow-time-scale OLTC and a fast-time-scale PV inverter;

[0009] Multi-agent voltage control Markov process modeling for dual-time-scale voltage control process;

[0010] Based on the multi-agent voltage-controlled Markov process, an improved MASAC algorithm is used to train the dual-time-scale agent;

[0011] The denoising autoencoder DAE measurement completion model based on graph convolutional neural network GCN is used to generate complete global voltage information, and the improved MASAC algorithm is used to generate the action of the control device.

[0012] As a further optimization scheme of the reactive power voltage control method of a distribution network in a measurement-missing environment described in the present invention, a distribution network environment with a large-scale measurement-missing environment is constructed as follows:

[0013] Measuring equipment is installed at the root node of the distribution network, the initial node of the switching branch and the access node of the photovoltaic unit to construct a distribution network environment with large-scale missing measurements.

[0014] As a further optimization scheme of the reactive power voltage control method of the distribution network in a measurement-missing environment described in the present invention, the GCN encoder is used to mine the spatial correlation of various finite physical quantities, as follows:

[0015] In non-Euclidean space, a radial distribution network with a substation is regarded as an undirected graph G consisting of N nodes and N-1 edges, denoted by G = (V, E), where V and E represent the set of nodes and edges respectively. The node feature matrix X and the adjacency matrix A of the undirected graph G are used as the input of GCN. The forward propagation process of the GCN encoder is recorded as:

[0016]

[0017] Among them, n is the original dimension of the node feature; is the original characteristic matrix of the distribution network diagram; is the adjacency matrix considering self-correlation; is the angle matrix; is the trainable parameter of layer l, Determines the weight of the edge between nodes; σ(·) is the activation function, is the original feature matrix, is the l+1 layer feature matrix, is the l-layer feature matrix.

[0018] As a further optimization scheme of the reactive voltage control method of a distribution network in a measurement missing environment described in the present invention, a DNN decoder is used to restore the voltage information of the measurement missing node, as follows:

[0019] DNN is composed of multiple layers of fully connected perceptrons, and its forward propagation process is recorded as:

[0020]

[0021] Among them, M (l) is the output of the first layer, and the output of the last layer is the complete voltage measurement information, M (l+1) is the output of layer l+1; and b (l) are the training weights and biases of layer l.

[0022] As a further optimization scheme of the reactive power voltage control method of the distribution network in a measurement-missing environment described in the present invention, a noise reduction autoencoder DAE measurement completion model based on a graph convolutional neural network GCN is established, which is as follows:

[0023] Step 2.3.1: Add multiplicative noise to the complete historical data to simulate the measurement missing environment:

[0024] The mask matrix is ​​multiplied by the complete historical active, reactive and voltage data to simulate the measurement missing environment, as shown in formula (3):

[0025]

[0026] Where C is a multiplicative noise matrix containing N elements arranged in the order of node numbers. If the i-th node is equipped with a measurement device, then the i-th element C in C is i =1, otherwise C i =0; X is the node feature matrix after masking, U e , P e and Q e are the voltage, net active power and net reactive power of the e-th node, 1≤e≤N;

[0027] Step 2.3.2: Perform forward propagation based on the principles of GCN and DNN to determine the loss function:

[0028] Input X and the adjacent matrix A into the two-layer graph convolutional neural network encoder, and perform forward propagation according to the principle of formula (1) to obtain the node feature matrix H after denoising and aggregating neighborhood features out , H out Flatten it into a one-dimensional vector and input it into the DNN decoder. Perform forward propagation according to the principle of formula (2) to obtain the complete node voltage output M out，n is the complement voltage of the nth node; the loss function takes the mean square error MSE between the true voltage and the complement voltage:

[0029]

[0030] Among them, U n is the actual voltage of the nth node;

[0031] During the training process, the denoising autoencoder based on the graph convolutional neural network uses the gradient descent algorithm to update the network parameters of each layer of the encoder and decoder according to the loss function.

[0032] As a further optimization scheme of the reactive power voltage control method of a distribution network in a measurement-missing environment described in the present invention, a dual-time-scale voltage control model is established in a distribution network environment with a large-scale measurement-missing environment. The control device in the dual-time-scale voltage control model includes a slow-time-scale OLTC and a fast-time-scale PV inverter, as follows:

[0033] (1) Objective function

[0034]

[0035] Among them, T s is the number of slow time scale cycles per day, T f is the number of fast time scale cycles contained in a slow time scale, T and t are the numbers of the slow time scale and the fast time scale respectively, represents the voltage of the i-th node at the slow time scale T and the fast time scale t;

[0036] (2) Distflow constraints of distribution network

[0037]

[0038]

[0039]

[0040] U min 2 ≤U i，t ≤U max 2 (9)

[0041] Among them, P ij,t and Q ij,t are respectively the active and reactive power on the branch from the i-th node to the j-th node; F(i) is the set of end nodes of the branch with the i-th node as the first end node; T(i) is the set of head nodes of the branch with the i-th node as the end node; l ij,t is the square of the current amplitude on the branch from the i-th node to the j-th node; is the predicted value of photovoltaic active output of the i-th node; is the reactive power output of the photovoltaic inverter at a fast time scale; U i,t and U j,t are the squares of the voltage amplitudes at the i-th node and the j-th node respectively; N S is the set of system nodes; BS is the system branch set; P i,t and Q i,t are respectively the active and reactive power injection of the i-th node; P ki,t and Q ki,t are respectively the active and reactive power on the path from the kth node to the ith node; l ki,t is the square of the current amplitude on the branch from the kth node to the jth node; r ki and x ki are the branch resistance and reactance from the kth node to the jth node respectively; r ij and x ij are the branch resistance and reactance from the i-th node to the j-th node respectively; U min and U max are the upper and lower limits of node voltage respectively;

[0042] (3) OLTC and PV unit constraints

[0043]

[0044]

[0045] in, is the square of the root node voltage of the distribution network, tap T is the OLTC gear at the slow time scale T, ΔU is the normalized voltage adjustable for each gear; k is the PV unit number, is the photovoltaic active output value, is the upper limit of photovoltaic reactive power output, is the photovoltaic reactive power output value, is the photovoltaic reactive power output boundary value, is the photovoltaic installed capacity, and m is the capacity elasticity coefficient.

[0046] As a further optimization scheme of the reactive power voltage control method of the distribution network in the measurement missing environment described in the present invention, a multi-agent voltage control Markov process modeling is performed on the dual-time scale voltage control process, specifically as follows: (1) Slow time scale agent

[0047] State: The global voltage situation at the beginning of each slow time scale:

[0048]

[0049] in, is the voltage value of each node, 1≤e≤N; S oltc,T is the global voltage observation at a slow time scale T;

[0050] Action: OLTC actions are discrete actions;

[0051] Bonus: Sum of voltage deviations of fast time scales within each slow time scale;

[0052]

[0053] Among them, r T is the reward value at the slow time scale T, T f is the total number of fast time scale periods contained in a slow time scale;

[0054] (2) Fast-time-scale agents

[0055] Status: Global voltage status at each fast time scale:

[0056]

[0057] Among them, S pv,t For fast time scale global voltage observation;

[0058] Action: The action of the PV inverter is continuous action, recorded as:

[0059]

[0060] in, is the action of the vth PV inverter, 1≤v≤Npv, Npv is the total number of PV inverters;

[0061] Bonus: Voltage deviation at each fast time scale:

[0062]

[0063] Among them, r t is the reward value of fast time t.

[0064] As a further optimization scheme of the reactive voltage control method of the distribution network in a measurement-missing environment described in the present invention, based on the multi-agent voltage control Markov process, an improved MASAC algorithm is used to train the dual-time-scale agent; the details are as follows:

[0065] (1) The goal of the improved MASAC algorithm is to maximize the linear combination of the expected return and entropy of the strategy:

[0066]

[0067] Among them, H(π(·|s t )) is the strategy entropy value, α is the temperature coefficient of entropy, D is the experience replay pool, γ is the discount coefficient, J(π) algorithm objective function, is the expected operation of the algorithm objective function, r(s t ,a t) is the reward value at the current moment; π(·|s t ) represents the probability of selecting each action in the current state;

[0068] (2) Action value function and state value function of the improved MASAC algorithm:

[0069]

[0070]

[0071] Among them, Q(s t ,a t ) is the action value function, is the expected Q value calculation at the next moment, π(a t ∣s t ) is the policy function, V(s t ) is the state value function, Calculate the expected value of the state at the next moment;

[0072] (3) The probability distribution of the optimal strategy is approximated by KL divergence:

[0073]

[0074] Among them, D KL is the KL divergence calculation, Z π (s,·) is the normalized function of the probability distribution, π' is the optimal strategy, π(·|s) is the strategy function, Q π (s,·) is the Q value under the current strategy;

[0075] (4) The loss function of the Q function is recorded as:

[0076]

[0077] Among them, ω is the parameter of the Q network, ω' is the parameter of the target network of the Q network; L Q (ω) is the loss function of the Q function, is the expected operation of the Q loss function, Q ω (s t ,a t ) is the Q value of the Q network at the current moment, r t is the reward for the current time step, is the Q value of the target network at the next moment;

[0078] During the training process, two Q networks are introduced to prevent overfitting, and the gradient descent algorithm is used to update the network parameters;

[0079] (5) The gradient of the policy function is recorded as:

[0080]

[0081] Among them, L π (θ) is the loss function of the policy function, is the expected operation of the policy loss function, αlogπ θ (a t ∣s t ) is the strategy entropy value.

[0082] The present invention also discloses a device for a reactive voltage control method of a distribution network in a measurement-missing environment, comprising a distribution network environment perception module with measurement missing, a measurement completion module and a dual-time-scale voltage control module; wherein,

[0083] The distribution network environment perception module for measuring large-scale missing distribution networks is used to collect and measure physical measurement information of the root node of the distribution network, the initial node of the switching branch, and the access node of the photovoltaic unit in the large-scale missing distribution network environment;

[0084] The measurement completion module uses the denoising autoencoder DAE based on the graph convolutional neural network GCN to mine the spatial correlation of each finite physical quantity and restore the missing node voltage information according to the physical measurement information in the measurement large-scale missing environment; the restored complete node voltage information is transmitted to the dual-time scale voltage control module;

[0085] The dual-time-scale voltage control module is used to obtain complete node voltage information as the slow-time-scale and fast-time-scale intelligent agent states, and to perform slow-time-scale OLTC and fast-time-scale PV inverter regulation, thereby realizing dual-time-scale voltage control.

[0086] As a further optimization scheme of the device of the reactive power voltage control method of the distribution network in the measurement missing environment described in the present invention,

[0087] The distribution network environment perception module, which is missing in a large area of ​​measurement, collects voltage, active and reactive data of limited measurement nodes through smart meters, organizes the limited physical measurement information and transmits it to the measurement completion module;

[0088] The measurement completion module uses the trained GCN encoder to mine the spatial correlation of each finite physical quantity, uses the DNN decoder to restore the voltage information of the missing measurement nodes, and passes it to the dual-time scale voltage control module;

[0089] The dual-time-scale voltage control module uses the complete node voltage information as the agent observation and uses the improved MASAC algorithm to control the slow-time-scale OLTC gear and the fast-time-scale PV inverter reactive output power to achieve dual-time-scale model-free voltage control.

[0090] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0091] Aiming at the problem of reactive voltage control in distribution networks in the absence of measurement, the present invention proposes a multi-agent voltage control method combining deep learning and deep reinforcement learning; a denoising autoencoder based on a graph convolutional neural network is constructed to complete the global voltage measurement, and a deep reinforcement learning algorithm is used to control the accurate action of each device. The invention can generate an accurate global state of the system based on limited measurement information and output an accurate control strategy, reducing the deployment cost of communication and full-node measurement facilities and realizing model-free voltage control. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Figure 1 is a flow chart of a multi-agent voltage control method based on robust deep reinforcement learning of the present invention;

[0093] Figure 2 Schematic diagram of the topological structure of the distribution network used in the test example of the present invention;

[0094] Figure 3 are the training set load and photovoltaic output data of the test example adopted by the present invention; wherein (a) is the photovoltaic output curve, and (b) is the load curve;

[0095] Figure 4 It is the test set load and photovoltaic output data of the test example adopted by the present invention; wherein, (a) is the photovoltaic output curve, and (b) is the load curve. DETAILED DESCRIPTION

[0096] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0097] like Figure 1 FIG. 1 is a flow chart of a method for controlling reactive voltage of a distribution network in a measurement-missing environment according to the present invention, comprising the following steps:

[0098] Step 1: Build a distribution network environment with large-scale missing measurements;

[0099] Step 2: Establish a GCN-based denoising autoencoder measurement completion model, which includes a GCN encoder and a DNN decoder. The GCN encoder mines the spatial correlation of each finite physical quantity, and the DNN decoder restores the voltage information of the missing nodes in the measurement.

[0100] Step 2.1, establish a GCN encoder to mine the spatial correlation of various finite physical quantities in the distribution network;

[0101] Step 2.2, construct a DNN decoder to restore the missing node voltage information;

[0102] Step 2.3, build a complete model of denoising autoencoder based on graph convolutional neural network;

[0103] Step 2.3.1, add multiplicative noise to the complete historical data to simulate the measurement missing environment;

[0104] Step 2.3.2, perform forward propagation based on the GCN and DNN principles to determine the loss function;

[0105] Step 3, establish a dual-time-scale voltage control model in a measurement-missing environment, where the control devices in the model include a slow-time-scale OLTC and a fast-time-scale PV inverter;

[0106] Step 4, modeling the dual-time-scale voltage control process using a multi-agent fully observable Markov process;

[0107] Step 5, based on the constructed multi-agent voltage-controlled Markov process, the improved MASAC algorithm is used to train the dual-time-scale agent;

[0108] In step 6, the proposed control method is applied to the actual distribution network environment with large-scale missing measurements. First, the GCN-based DAE is used to generate complete global voltage information, and then the improved MASAC algorithm is used to control the actions of each device.

[0109] The specific steps mentioned above are as follows:

[0110] Step 1: Build a distribution network environment with large-scale missing measurements:

[0111] Due to the channel transmission pressure and the difficulty of deploying full-node measurement equipment, the control center cannot obtain global status information during the actual operation and control of the distribution network. The measurement constructed by the present invention lacks a large-scale distribution network environment, and the measurement equipment is only installed at the root node, the initial node of the switching branch, and the photovoltaic access node. The measurement coverage of the entire system is 30%. The control center needs to quickly generate the voltage value of the node without measurement equipment based on the given limited real-time measurement information, and transmit the global observation results to the voltage control agent.

[0112] Step 2: Establish a GCN-based denoising autoencoder measurement completion model, including the GCN encoder mining the spatial correlation of each finite physical quantity and the DNN decoder restoring the missing node voltage information:

[0113] Step 2.1, establish a GCN encoder to mine the spatial correlation of various finite physical quantities in the distribution network:

[0114] In non-Euclidean space, a radial distribution network with a substation is regarded as an undirected graph G consisting of N nodes and N-1 edges, denoted by G = (V, E), where V and E represent the set of nodes and edges respectively. The node feature matrix X and the adjacency matrix A of the undirected graph G are used as the input of GCN. The forward propagation process of the GCN encoder is recorded as:

[0115]

[0116] Among them, n is the original dimension of the node feature; is the original characteristic matrix of the distribution network diagram; is the adjacency matrix considering self-correlation; is the angle matrix; is the trainable parameter of layer l, Determines the weight of the edge between nodes; σ(·) is the activation function, is the original feature matrix, is the l+1 layer feature matrix, is the l-layer feature matrix.

[0117] Step 2.2: Build a DNN decoder to restore the missing node voltage information:

[0118] DNN is composed of multiple layers of fully connected perceptrons, and its forward propagation process is recorded as:

[0119]

[0120] Among them, M (l) is the output of the first layer, and the output of the last layer is the complete voltage measurement information, M (l+1) is the output of layer l+1; and b (l) are the training weights and biases of layer l.

[0121] Step 2.3, build a complete model of denoising autoencoder based on graph convolutional neural network:

[0122] Step 2.3.1, add multiplicative noise to the complete historical data to simulate the measurement missing environment:

[0123] The mask matrix is ​​multiplied by the complete historical active, reactive and voltage data to simulate the measurement missing environment, as shown in formula (3):

[0124]

[0125] Where C is a multiplicative noise matrix containing N elements arranged in the order of node numbers. If the i-th node is equipped with a measurement device, then the i-th element C in C is i =1, otherwise C i=0; X is the node feature matrix after masking, U e , P e and Q e are the voltage, net active power and net reactive power of the e-th node respectively, 1≤e≤N.

[0126] Step 2.3.2, perform forward propagation based on the GCN and DNN principles to determine the loss function:

[0127] Input X and the adjacent matrix A into the two-layer graph convolutional neural network encoder, and perform forward propagation according to the principle of formula (1) to obtain the node feature matrix H after denoising and aggregating neighborhood features out , H out Flatten it into a one-dimensional vector and input it into the DNN decoder. Perform forward propagation according to the principle of formula (2) to obtain the complete node voltage output M out，n is the complement voltage of the nth node; the loss function takes the mean square error MSE between the true voltage and the complement voltage:

[0128]

[0129] During the training process, the denoising autoencoder based on the graph convolutional neural network uses the gradient descent algorithm to update the network parameters of each layer of the encoder and decoder according to the loss function.

[0130] Step 3: Establish a dual-time-scale voltage control model in a measurement-missing environment. The control equipment includes a slow-time-scale OLTC and a fast-time-scale PV inverter:

[0131] (1) Objective function

[0132]

[0133] Among them, T s is the number of slow time scale cycles per day, T f is the number of fast time scale cycles contained in a slow time scale, T and t are the numbers of the slow time scale and the fast time scale respectively, Represents the voltage of the i-th node at the slow time scale T and the fast time scale t.

[0134] (2) Distflow constraints of distribution network

[0135]

[0136]

[0137]

[0138] U min 2 ≤U i，t ≤Umax 2 (9)

[0139] Among them, P ij,t and Q ij,t are respectively the active and reactive power on the branch from the i-th node to the j-th node; F(i) is the set of end nodes of the branch with the i-th node as the first end node; T(i) is the set of head nodes of the branch with the i-th node as the end node; l ij,t is the square of the current amplitude on the branch from the i-th node to the j-th node; is the predicted value of photovoltaic active output of the i-th node; is the reactive power output of the photovoltaic inverter at a fast time scale; U i,t and U j,t are the squares of the voltage amplitudes at the i-th node and the j-th node respectively; N S is the set of system nodes; B S is the system branch set; P i,t and Q i,t are respectively the active and reactive power injection of the i-th node; P ki,t and Q ki,t are respectively the active and reactive power on the path from the kth node to the ith node; l ki,t is the square of the current amplitude on the branch from the kth node to the jth node; r ki and x ki are the branch resistance and reactance from the kth node to the jth node respectively; r ij and x ij are the branch resistance and reactance from the i-th node to the j-th node respectively; U min and U max are the upper and lower limits of the node voltage respectively.

[0140] (3) OLTC and PV unit constraints

[0141]

[0142]

[0143] in, is the square of the root node voltage of the distribution network, tap T is the OLTC gear at the slow time scale T, ΔU is the normalized voltage adjustable for each gear; k is the PV unit number, is the photovoltaic active output value, is the upper limit of photovoltaic reactive power output, is the photovoltaic reactive power output value, is the photovoltaic reactive power output boundary value, is the photovoltaic installed capacity, and m is the capacity elasticity coefficient.

[0144] Step 4, modeling the dual-time-scale voltage control process using a multi-agent fully observable Markov process;

[0145] The multi-agent Markov process of OLTC and PV inverter agents is essentially a multi-agent cooperation process. The cooperation between agents is reflected in the fact that the reward value of each slow time scale step is the sum of the fast time scale rewards within the slow time scale.

[0146] (1) Slow-time-scale agents

[0147] State: The global voltage situation at the beginning of each slow time scale:

[0148]

[0149] in, is the voltage value of each node, 1≤e≤N; S oltc,T is the global voltage observation at a slow time scale T;

[0150] Action: OLTC actions are discrete actions;

[0151] Bonus: Sum of voltage deviations of fast time scales within each slow time scale;

[0152]

[0153] Among them, r T is the reward value at the slow time scale T, T f is the total number of fast time scale periods contained in a slow time scale;

[0154] (2) Fast-time-scale agents

[0155] Status: Global voltage status at each fast time scale:

[0156]

[0157] Among them, S pv,t For fast time scale global voltage observation;

[0158] Action: The action of the PV inverter is continuous action, recorded as:

[0159]

[0160] in, is the action of the vth PV inverter, 1≤v≤Npv, Npv is the total number of PV inverters;

[0161] Bonus: Voltage deviation at each fast time scale:

[0162]

[0163] Among them, r t is the reward value of fast time t;

[0164] Step 5: Based on the constructed multi-agent voltage-controlled Markov process, the improved MASAC algorithm is used to train the dual-time-scale agent.

[0165] (1) The goal of the algorithm is to maximize the linear combination of the expected return and entropy of the strategy:

[0166]

[0167] Among them, H(π(·|s t )) is the strategy entropy value, α is the temperature coefficient of entropy, D is the experience replay pool, γ is the discount coefficient, J(π) algorithm objective function, is the expected operation of the algorithm objective function, r(s t ,a t ) is the reward value at the current moment; π(·|s t ) represents the probability of selecting each action in the current state.

[0168] (2) Action value function and state value function of the algorithm:

[0169]

[0170]

[0171] Among them, Q(s t ,a t ) is the action value function, is the expected Q value calculation at the next moment, π(a t ∣s t ) is the policy function, V(s t ) is the state value function, Calculate the expected value of the state at the next moment

[0172] (3) The probability distribution of the optimal strategy is approximated by KL divergence:

[0173]

[0174] Among them, D KL is the KL divergence calculation, Z π (s,·) is the normalized function of the probability distribution, π' is the optimal strategy, π(·|s) is the strategy function, Q π (s,·) is the Q value under the current strategy.

[0175] (4) The loss function of the Q function is recorded as:

[0176]

[0177] Among them, ω is the parameter of the Q network, ω' is the parameter of the target network of the Q network; L Q (ω) is the loss function of the Q function, is the expected operation of the Q loss function, Q ω (s t ,a t ) is the Q value of the Q network at the current moment, r t is the reward for the current time step, is the Q value of the target network at the next moment. Two Q networks are introduced during the training process to prevent overfitting, and the gradient descent algorithm is used to update the network parameters.

[0178] (5) The gradient of the policy function is recorded as:

[0179]

[0180] Among them, L π (θ) is the loss function of the policy function, is the expected operation of the policy loss function, αlogπ θ (a t ∣s t ) is the strategy entropy value.

[0181] In step 6, the proposed control method is applied to the actual distribution network environment with large-scale missing measurements. First, the GCN-based DAE is used to generate complete global voltage information, and then the improved MASAC algorithm is used to control the actions of each device.

[0182] The present invention also provides a device for a reactive voltage control method of a distribution network in a measurement-missing environment, comprising a distribution network environment perception module with measurement missing, a measurement completion module and a dual-time-scale voltage control module; wherein,

[0183] The distribution network environment perception module for measuring large-scale missing distribution networks is used to collect and measure physical measurement information of the root node of the distribution network, the initial node of the switching branch, and the access node of the photovoltaic unit in the large-scale missing distribution network environment;

[0184] The measurement completion module uses the denoising autoencoder DAE based on the graph convolutional neural network GCN to mine the spatial correlation of each finite physical quantity and restore the voltage information of the missing measurement nodes according to the physical measurement information in the environment where a large range of measurements are missing.

[0185] The restored complete node voltage information is transmitted to the dual-time-scale voltage control module;

[0186] The dual-time-scale voltage control module is used to obtain complete node voltage information as the slow-time-scale and fast-time-scale intelligent agent states, and to perform slow-time-scale OLTC and fast-time-scale PV inverter regulation, thereby realizing dual-time-scale voltage control.

[0187] The distribution network environment perception module, which is missing in a large range of measurements, collects voltage, active and reactive data of limited measurement nodes through smart meters, organizes the limited physical measurement information and transmits it to the measurement completion module.

[0188] The measurement completion module uses the trained GCN encoder to mine the spatial correlation of each finite physical quantity, uses the DNN decoder to restore the voltage information of the missing measurement nodes, and passes it to the dual-time scale voltage control module;

[0189] The dual-time-scale voltage control module uses the complete node voltage information as the agent observation and uses the improved MASAC algorithm to control the slow-time-scale OLTC gear and the fast-time-scale PV inverter reactive output power to achieve dual-time-scale model-free voltage control.

[0190] The algorithm can be implemented on the 12th generation Intel(R) Core(TM) i7-12700H CPU based on the PyTorch framework in Python 3.7. Figure 2 The effectiveness of the present invention is verified by taking the multi-feeder distribution network shown as an example. The OLTC operates once every half an hour, and the PV inverter on the feeder operates once every 5 minutes. There are 11 measurement nodes in total, including the root node, the switch branch head end node and the photovoltaic access node. Figure 3 The training set load and photovoltaic output curve for the test case are: Figure 3 (a) is the photovoltaic output curve. Figure 3 (b) in the figure is the load curve. Figure 4 The test set load and photovoltaic output curve for the test case, Figure 4 (a) is the photovoltaic output curve. Figure 4 (b) in the figure is the load curve.

[0191] Table 1 shows the comparison of voltage control effect between this model control method and other algorithms

[0192] Table 1 Voltage control effects of various algorithms

[0193]

[0194] The results show that the control effect of the MASAC algorithm, which uses the voltage information restored by the denoising autoencoder based on the graph convolutional neural network as the global measurement, is almost completely consistent with the control effect of the MASAC algorithm under ideal conditions and is close to the global optimum. The average voltage deviation of the three MASAC algorithms is significantly lower than that of the MADDPG algorithm. The reason is that the MASAC algorithm is based on the maximum entropy training framework, which balances the global and local exploration capabilities of the agent during the adaptive temperature coefficient update process, and enhances the exploration of high-dimensional space.

[0195] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0196] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0197] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0198] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0199] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0200] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for controlling reactive power voltage of a distribution network in a measurement-deficient environment, characterized in that: include: Construct a distribution network environment with large-scale missing measurements; A denoising autoencoder DAE measurement completion model based on graph convolutional neural network GCN is established, wherein the denoising autoencoder measurement completion model includes a GCN encoder and a DNN decoder, wherein the GCN encoder is used to mine the spatial correlation of each finite physical quantity, and the DNN decoder is used to restore the voltage information of the missing measurement node; Establishing a dual-time-scale voltage control model in a distribution network environment with large-scale measurement loss, wherein the control devices in the dual-time-scale voltage control model include a slow-time-scale OLTC and a fast-time-scale PV inverter; Multi-agent voltage control Markov process modeling for dual-time-scale voltage control process; Based on the multi-agent voltage-controlled Markov process, an improved MASAC algorithm is used to train the dual-time-scale agent; The denoising autoencoder DAE measurement completion model based on graph convolutional neural network GCN is used to generate complete global voltage information, and the improved MASAC algorithm is used to generate the action of the control device.

2. The method for controlling reactive power voltage of a distribution network in a measurement-missing environment according to claim 1, characterized in that: The specific construction of the distribution network environment with large-scale missing measurements is as follows: Measuring equipment is installed at the root node of the distribution network, the initial node of the switching branch, and the access node of the photovoltaic unit to construct a distribution network environment with large-scale missing measurements.

3. The method for controlling reactive power voltage of a distribution network in a measurement-missing environment according to claim 1, characterized in that: The GCN encoder is used to mine the spatial correlation of various finite physical quantities, as follows: In non-Euclidean space, a radial distribution network with a substation is regarded as an undirected graph G consisting of N nodes and N-1 edges, denoted by G = (V, E), where V and E represent the set of nodes and edges respectively. The node feature matrix X and the adjacency matrix A of the undirected graph G are used as the input of GCN. The forward propagation process of the GCN encoder is recorded as: Among them, n is the original dimension of the node feature; is the original characteristic matrix of the distribution network diagram; is the adjacency matrix considering self-correlation; is the angle matrix; is the trainable parameter of layer l, Determines the weight of the edge between nodes; σ(·) is the activation function, is the original feature matrix, is the l+1 layer feature matrix, is the l-layer feature matrix.

4. The method for controlling reactive power voltage of a distribution network in a measurement-missing environment according to claim 3, characterized in that: The DNN decoder is used to restore the missing node voltage information as follows: DNN is composed of multiple layers of fully connected perceptrons, and its forward propagation process is recorded as: Among them, M (l) is the output of the first layer, and the output of the last layer is the complete voltage measurement information, M (l+1) is the output of layer l+1; and b (l) are the training weights and biases of layer l.

5. The method for controlling reactive power voltage of a distribution network in a measurement-missing environment according to claim 4, characterized in that: A denoising autoencoder DAE measurement completion model based on graph convolutional neural network GCN is established as follows: Step 2.3.1: Add multiplicative noise to the historical complete data to simulate the measurement missing environment: The mask matrix is ​​multiplied by the complete historical active, reactive and voltage data to simulate the measurement missing environment, as shown in formula (3): Where C is a multiplicative noise matrix containing N elements arranged in the order of node numbers. If the i-th node is equipped with a measurement device, then the i-th element C in C is i =1, otherwise C i =0; X is the node feature matrix after masking, U e , P e and Q e are the voltage, net active power and net reactive power of the e-th node, 1≤e≤N; Step 2.3.2: Perform forward propagation based on the principles of GCN and DNN to determine the loss function: Input X and the adjacent matrix A into the two-layer graph convolutional neural network encoder, and perform forward propagation according to the principle of formula (1) to obtain the node feature matrix H after denoising and aggregating neighborhood features out , H out Flatten it into a one-dimensional vector and input it into the DNN decoder. Perform forward propagation according to the principle of formula (2) to obtain the complete node voltage output M out，n is the complement voltage of the nth node; the loss function takes the mean square error MSE between the true voltage and the complement voltage: Among them, U n is the actual voltage of the nth node; During the training process, the denoising autoencoder based on the graph convolutional neural network uses the gradient descent algorithm to update the network parameters of each layer of the encoder and decoder according to the loss function.

6. The method for controlling reactive power voltage of a distribution network in a measurement-missing environment according to claim 4, characterized in that: A dual-time-scale voltage control model is established in a distribution network environment with large-scale measurement loss. The control devices in the dual-time-scale voltage control model include a slow-time-scale OLTC and a fast-time-scale PV inverter, as follows: (1) Objective function Among them, T s is the number of slow time scale cycles per day, T f is the number of fast time scale cycles contained in a slow time scale, T and t are the numbers of the slow time scale and the fast time scale respectively, represents the voltage of the i-th node at the slow time scale T and the fast time scale t; (2) Distflow constraints of distribution network IN min 2 ≤U i,t ≤U max 2 (9) Among them, P ij,t and Q ij,t are respectively the active and reactive power on the branch from the i-th node to the j-th node; F(i) is the set of end nodes of the branch with the i-th node as the first end node; T(i) is the set of head nodes of the branch with the i-th node as the end node; l ij,t is the square of the current amplitude on the branch from the i-th node to the j-th node; is the predicted value of photovoltaic active output of the i-th node; is the reactive power output of the photovoltaic inverter at a fast time scale; U i,t and U j,t are the squares of the voltage amplitudes at the i-th node and the j-th node respectively; N S is the set of system nodes; B S is the system branch set; P i,t and Q i,t are respectively the active and reactive power injection of the i-th node; P ki,t and Q ki,t are respectively the active and reactive power on the path from the kth node to the ith node; l ki,t is the square of the current amplitude on the branch from the kth node to the jth node; r ki and x ki are the branch resistance and reactance from the kth node to the jth node respectively; r ij and x ij are the branch resistance and reactance from the i-th node to the j-th node respectively; U min and U max are the upper and lower limits of node voltage respectively; (3) OLTC and PV unit constraints in, is the square of the root node voltage of the distribution network, tap T is the OLTC gear at the slow time scale T, ΔU is the normalized voltage adjustable for each gear; k is the PV unit number, is the photovoltaic active output value, is the upper limit of photovoltaic reactive power output, is the photovoltaic reactive power output value, is the photovoltaic reactive power output boundary value, is the photovoltaic installed capacity, and m is the capacity elasticity coefficient.

7. The method for controlling reactive power voltage of a distribution network in a measurement-missing environment according to claim 4, characterized in that: The multi-agent voltage control Markov process modeling is carried out for the dual-time-scale voltage control process, as follows: (1) Slow time-scale agent State: The global voltage situation at the beginning of each slow time scale: in, is the voltage value of each node, 1≤e≤N; S oltc,T is the global voltage observation at a slow time scale T; Action: OLTC actions are discrete actions; Bonus: Sum of voltage deviations of fast time scales within each slow time scale; Among them, r T is the reward value at the slow time scale T, T f is the total number of fast time scale periods contained in a slow time scale; (2) Fast-time-scale agents Status: Global voltage status at each fast time scale: Among them, S pv,t For fast time scale global voltage observation; Action: The action of the PV inverter is continuous action, recorded as: in, is the action of the vth PV inverter, 1≤v≤Npv, Npv is the total number of PV inverters; Bonus: Voltage deviation at each fast time scale: Among them, r t is the reward value of fast time t.

8. The method for controlling reactive power voltage of a distribution network in a measurement-missing environment according to claim 1, characterized in that: Based on the multi-agent voltage-controlled Markov process, the improved MASAC algorithm is used to train the dual-time-scale agent; the details are as follows: (1) The goal of the improved MASAC algorithm is to maximize the linear combination of the expected return and entropy of the strategy: Among them, H(π(·|s t )) is the strategy entropy value, α is the temperature coefficient of entropy, D is the experience replay pool, γ is the discount coefficient, J(π) algorithm objective function, is the expected operation of the algorithm objective function, r(s t ,a t ) is the reward value at the current moment; π(·|s t ) represents the probability of selecting each action in the current state; (2) Action value function and state value function of the improved MASAC algorithm: Among them, Q(s t ,a t ) is the action value function, is the expected Q value calculation at the next moment, π(a t ∣s t ) is the policy function, V(s t ) is the state value function, Calculate the expected value of the state at the next moment; (3) The probability distribution of the optimal strategy is approximated by KL divergence: Among them, D KL is the KL divergence calculation, Z π (s,·) is the normalized function of the probability distribution, π' is the optimal strategy, π(·|s) is the strategy function, Q π (s,·) is the Q value under the current strategy; (4) The loss function of the Q function is recorded as: Among them, ω is the parameter of the Q network, ω' is the parameter of the target network of the Q network; L Q (ω) is the loss function of the Q function, is the expected operation of the Q loss function, Q ω (s t ,a t ) is the Q value of the Q network at the current moment, r t is the reward for the current time step, is the Q value of the target network at the next moment; During the training process, two Q networks are introduced to prevent overfitting, and the gradient descent algorithm is used to update the network parameters; (5) The gradient of the policy function is recorded as: Among them, L π (θ) is the loss function of the policy function, is the expected operation of the policy loss function, αlogπ θ (a t ∣s t ) is the strategy entropy value.

9. A device for controlling reactive power voltage of a distribution network in a measurement-missing environment, characterized in that: It includes a distribution network environment perception module with missing measurements, a measurement completion module and a dual-time scale voltage control module; among them, The distribution network environment perception module for measuring large-scale missing distribution networks is used to collect and measure physical measurement information of the root node of the distribution network, the initial node of the switching branch, and the access node of the photovoltaic unit in the large-scale missing distribution network environment; The measurement completion module uses the denoising autoencoder DAE based on the graph convolutional neural network GCN to mine the spatial correlation of each finite physical quantity and restore the missing node voltage information according to the physical measurement information in the measurement large-scale missing environment; the restored complete node voltage information is transmitted to the dual-time scale voltage control module; The dual-time-scale voltage control module is used to obtain complete node voltage information as the slow-time-scale and fast-time-scale intelligent agent states, and to perform slow-time-scale OLTC and fast-time-scale PV inverter regulation, thereby realizing dual-time-scale voltage control.

10. The device of the reactive power voltage control method of the distribution network in the measurement missing environment according to claim 9, characterized in that: The distribution network environment perception module, which is missing in a large area of ​​measurement, collects voltage, active and reactive data of limited measurement nodes through smart meters, organizes the limited physical measurement information and transmits it to the measurement completion module; The measurement completion module uses the trained GCN encoder to mine the spatial correlation of each finite physical quantity, uses the DNN decoder to restore the voltage information of the missing measurement nodes, and passes it to the dual-time scale voltage control module; The dual-time-scale voltage control module uses the complete node voltage information as the agent observation and uses the improved MASAC algorithm to control the slow-time-scale OLTC gear and the fast-time-scale PV inverter reactive output power to achieve dual-time-scale model-free voltage control.

Images