Model establishment method for distribution network node voltage control mode selection, dynamic selection method for distribution network node voltage control mode, control system and power system

Abstract

This application provides a model building method for selecting voltage control modes at distribution network nodes, a dynamic selection method for voltage control modes at distribution network nodes, a control system, and a power system. The model building method includes: collecting voltage characteristics of each node in the distribution network at a certain moment; calculating the weights of each node using an attention mechanism based on the voltage characteristics, and using the weight calculation results as attribute values; inputting the attribute values ​​into a first deep reinforcement learning model for training to obtain the frequency of occurrence of each node when the model reaches the optimal target state; creating frequency curves of occurrence of each node at continuous moments based on the frequency of occurrence of each node when the model reaches the optimal target state at multiple moments; and matching the corresponding node voltage control mode based on the frequency curves of occurrence of each node. This method comprehensively considers the degree of mutual influence between the voltage characteristics of each node, realizes the comprehensive determination of the voltage control mode of distribution network nodes, and is applicable to the power system field.

Description

Technical Field

[0001] This application relates to the field of power systems, specifically to a model establishment method for selecting voltage control modes at distribution network nodes, a dynamic selection method for voltage control modes at distribution network nodes, a control system, and a power system. Background Technology

[0002] In power systems, controlling the node voltage of the distribution network is a core measure to ensure the safe, reliable, and economical operation of the power system. For today's new distribution networks, voltage control faces greater challenges compared to traditional distribution networks, mainly in the following aspects: First, traditional distribution networks have a radial topology with a single power flow direction, and their voltage control relies primarily on centralized control of the voltage at upstream nodes. However, new distribution networks, due to the integration of numerous renewable energy sources, combine centralized and distributed (such as rooftop photovoltaics) power flows, resulting in variable power flow directions. Therefore, relying solely on centralized control at upstream nodes cannot achieve voltage stability. Second, traditional distribution network nodes have low levels of automation and incomplete telemetry and remote signaling functions, making it difficult to provide effective data for analysis when formulating voltage control strategies. Therefore, voltage control in traditional distribution networks generally does not consider the participation of nodes in medium and low voltage distribution networks (10kV and below). In contrast, voltage control in new distribution networks needs to cover all voltage levels, including high voltage (110kV, 35kV), medium voltage (10kV), and low voltage (380V / 220V), requiring higher standards for voltage control optimization strategies and necessitating coordinated cooperation among networks at various voltage levels.

[0003] Because new distribution networks connect to a wider variety of load types, including distributed photovoltaic (PV) systems, microgrids, smart parks, smart distribution areas, and numerous controllable resources such as air conditioners and electric vehicles, they offer a broader voltage regulation space compared to traditional distribution networks. Therefore, the node voltage control in new distribution networks is no longer limited to the traditional single-phase control mode; local control has become one of the important selection modes. Furthermore, due to the integration of numerous new energy power plants, rooftop PV systems, electric vehicles, air conditioners, and other controllable load resources, new distribution networks possess significant voltage control potential, enabling regulation of voltage at medium and low voltage distribution network nodes. This allows them to play a crucial regulatory role in the research and formulation of voltage control strategies. Therefore, the main research direction for voltage control in new distribution networks lies in the coordinated control between high, medium, and low voltage distribution networks.

[0004] Currently, the basic methods of coordinated control mainly include centralized control, local control, and hierarchical control. Centralized control primarily achieves voltage balance in a regional distribution network by adjusting parameters such as reactive power, transformer tap position, and active power at important nodes. Local control mainly achieves voltage stability at nodes experiencing voltage exceedances by controlling their power output and adjusting transformer tap positions. Hierarchical control, encompassing both centralized and local control, primarily achieves voltage balance by adjusting nodes at different voltage levels and in different regions in a tiered manner. Regardless of the control mode adopted, determining the types of nodes requiring control and the corresponding control mode is crucial.

[0005] Currently, there are three main methods for selecting new distribution network node voltage control modes: First, determine the voltage-sensitive nodes of the regional distribution network based on voltage sensitivity, and achieve voltage balance of the regional power grid by controlling the node with the highest voltage sensitivity; second, control special nodes in the distribution network that frequently exceed voltage limits or have large regulation capabilities (usually selecting nodes such as active power transmission, reactive power imbalance, severe voltage exceedance, and new energy power plants for control) to achieve local balance of the nodes; third, use genetic optimization algorithms, particle swarm optimization algorithms, and other means to perform optimization calculations on the distribution network, set objective functions for optimal control, and finally generate node control schemes that meet the optimization objectives.

[0006] The three selection methods mentioned above all have significant drawbacks in practical applications: First, while the voltage sensitivity selection mechanism is the most common, voltage sensitivity calculation relies on power flow calculation. Power flow calculation in new distribution networks involves modeling new energy power station node models, load models, and other models, which is more difficult than traditional power flow calculation. Furthermore, voltage sensitivity cannot quickly adapt to changes in the power grid topology and operating parameters, leading to complex and inaccurate analysis results, and failing to provide a screening mechanism for selecting voltage control modes at distribution network nodes. Second, although selecting specific nodes for voltage control based on the cause of voltage exceedance is a major method used in actual power grid engineering, this method mainly focuses on the superficial manifestations of voltage exceedance, such as active power transmission and reactive power imbalance. Therefore, voltage regulation targeting these nodes is a coarse-grained approach. While node selection is relatively easy and control modes are varied, it ignores the overall internal mechanisms of the power grid, failing to achieve precise control of the distribution network and resulting in poor control performance. Third, many existing optimization algorithms typically only satisfy the power grid operation objective function. Although they cover multiple objectives such as voltage stability, active power reduction, and control cost, they neglect the selection of node control modes, making it difficult to achieve ideal results in applications. Summary of the Invention

[0007] To address one of the aforementioned technical deficiencies, this application provides a model establishment method for selecting voltage control modes at distribution network nodes, a dynamic selection method for voltage control modes at distribution network nodes, a control system, and a power system.

[0008] According to the first aspect of this application, a model establishment method for selecting the voltage control mode of a distribution network node is provided, comprising:

[0009] The voltage characteristics of each node in the distribution network at time t1 are collected. The voltage characteristics include: active power, reactive power, effective voltage value and voltage phase angle.

[0010] Based on the voltage characteristics of each node at time t1, an attention mechanism is used to calculate the weight of each node, and the weight calculation result is used as the attribute degree value to measure the relationship between the voltages of each node in the distribution network at time t1.

[0011] Based on the attribute degree values, construct the state space, action space, and reward function of the deep reinforcement learning model;

[0012] Each state space variable in the state space is trained one by one, so that the deep reinforcement learning model reaches the optimal target state during training and outputs the occurrence frequency of each node when it reaches the optimal target state.

[0013] The successfully trained deep reinforcement learning model will be used as the first deep reinforcement learning model for the final selection of voltage control modes at distribution network nodes.

[0014] Preferably, based on the voltage characteristics of each node at time t1, an attention mechanism is used to calculate the weights of each node, and the weight calculation results are used as attribute values ​​to measure the interrelationships between the voltages of each node in the distribution network at time t1. Specifically, this includes:

[0015] Based on the active power, reactive power, effective voltage value and voltage phase angle of each node in the distribution network at time t1, voltage feature matrices are established to represent the voltage change characteristics of each node in the distribution network at time t1.

[0016] Based on the voltage characteristic matrices, construct the node characteristic matrix of the distribution network at time t1;

[0017] An attention mechanism is used to calculate the similarity between the features of each node in the node feature matrix, and the attention mechanism matrix is ​​obtained based on the similarity calculation results.

[0018] After normalizing the node feature matrix, the attention weight matrix is ​​calculated based on the attention mechanism matrix and the normalized node feature matrix.

[0019] The attention weight matrix is ​​used as the attribute degree value to measure the relationship between the voltages of each node in the distribution network at time t1.

[0020] Preferably, the step of constructing the state space, action space, and reward function of the deep reinforcement learning model based on the attribute degree value specifically includes:

[0021] Based on the attention weight matrix, a state space of the deep reinforcement learning model is constructed, and the variables in the state space correspond one-to-one with the weight elements in the attention weight matrix.

[0022] Based on the mutual transition actions between state variables in the state space, the action space of the deep reinforcement learning model is constructed, wherein each transition action is executed by an ε-greedy policy.

[0023] Based on the attention weight matrix, the reward function of the deep reinforcement learning model is constructed as follows: when training a certain state space variable, if the weight element between the current state space variable and other state space variables is 0, the reward function value is 0; if the weight element between the current state space variable and other state space variables is not 0, the reward function value is the sum of the weight element between them and the next state space variable.

[0024] Preferably, the deep reinforcement learning model is a DQN network model, and the step of training each state space variable in the state space one by one, so that the deep reinforcement learning model reaches the optimal target state during training and outputs the occurrence frequency of each node when the optimal target state is reached, specifically includes:

[0025] Select a state space variable from the state space as the initial state space variable, input it into the training Q network of the first deep reinforcement learning model for training, and generate the corresponding initial training Q value.

[0026] The action space is executed according to the ε-greedy policy, the corresponding reward function value is calculated according to the reward function, and the action information of each action is used to form an experience dataset to build an experience pool.

[0027] Select at least one set of experience datasets from the experience pool and input them into the training Q network and target Q network of the first deep reinforcement learning model respectively, and calculate the training Q value of the training Q network and the target value of the target Q network respectively.

[0028] Determine whether the training Q-value has reached the target value. If it has, determine whether the model has reached the required number of iterations. If it has not reached the target value, select new initial state space variables and start a new round of training.

[0029] If the model reaches the required number of iterations, it is determined whether the model has performed target state calculation for each node; if the model has not reached the required number of iterations, the action space is executed according to the ε-greedy policy, and the next state space variable is reselected for training until the required number of iterations is reached.

[0030] If the model has performed target state calculation for each node, then the frequency of each node when the model reaches the optimal target state is statistically analyzed and output; if there are nodes that have not performed target state calculation, then the initial state space variables are reselected for a new round of training.

[0031] Preferably, the voltage feature matrix is ​​represented as:

[0032]

[0033] Where, x i.t1 Let p be the voltage characteristic matrix of the i-th node at time t1. i Let q be the active power of the i-th node. i Let U be the reactive power of the i-th node. i Let be the effective voltage value of the i-th node. Let be the voltage phase angle of the i-th node;

[0034] The node feature matrix is ​​represented as follows:

[0035]

[0036] Among them, X t1 Let n1 be the node characteristic matrix of the distribution network at time t1, where n1 is the number of nodes in the distribution network.

[0037] The attention mechanism matrix is ​​represented as follows:

[0038]

[0039] Among them, e t1 Let e ​​be the attention mechanism matrix at time t1. t1 The elements e in i.j Used to represent the node feature matrix X t1 Chinese x i.t1 and x j.t1 Similarity;

[0040] The element e i.j Represented as:

[0041]

[0042] Where ReLU is the activation function, a is the trainable parameter vector, W is the trainable parameter matrix used to perform linear transformation on the input features, || represents the concatenation of vectors, and k represents the number of node features contained in the node feature matrix, k=1,……,n1;

[0043] The normalized node feature matrix is ​​represented as follows: , The elements in Represented as:

[0044]

[0045] in, and The node feature matrix X is respectively t1 The mean and variance, 1 represents the variance control bias, and γ1 and β are the learnable parameters of the normalized computation layer in the dynamic convolutional neural network. Represents the Hadamard product;

[0046] The attention weight matrix is ​​represented as follows:

[0047]

[0048] Among them, W B.t1 Here is the attention weight matrix at time t1. For activation function, This is a similarity matrix. for The transpose of the matrix;

[0049] The similarity matrix Represented as:

[0050] .

[0051] Preferably, the state space is represented as s t1 The action space is represented as a t1 ;

[0052] The reward function is expressed as follows:

[0053]

[0054] Where, r i The reward function corresponding to training the i-th node is the weight element w between the i-th node and the j-th node. ij When the value is 0, the reward function value is 0. When the weight element w between node i and node j is 0... ij When the value is not 0, the reward function value is the weight element w. ijWith the next state space variable s j The sum of;

[0055] The ε-greedy strategy is expressed as follows:

[0056]

[0057] Wherein, π(a t1 |s t1 ) is the ε-greedy policy function; ε2 is the random selection probability, which takes the value ε2=max(0.01, 1-n2 / N), where n2 is the current training iteration and N is the total training iteration; argmaxQ(a t1 ,s t1 ) represents the maximum Q value corresponding to the optimal action; when the action is the optimal choice, its selection probability is 1-ε2, otherwise the selection probability is ε2.

[0058] Preferably, the empirical dataset is represented as follows:

[0059] (s) i a t1 r i s j )

[0060] Among them, s i Let a be the current state space variable being trained. t1 For the currently executing action, r i For the corresponding reward function value, s j The next state-space variable selected according to the state transition strategy;

[0061] When d sets of empirical datasets are selected from the experience pool and input into the training Q-network and target Q-network of the first deep reinforcement learning model respectively, d <d m , where d m The data capacity of the experience pool;

[0062] During training, the following formula is used to calculate the target Q-network:

[0063]

[0064] Where m is the current iteration number, T is the preset total number of iterations, and y m Let T be the target value of the Q network for the m-th training iteration. When m reaches T, the instantaneous reward r obtained by training the network Q is... m The reward value is γ2; when m does not reach T, the long-term cumulative reward value is the reward value; γ2 is the discount coefficient.

[0065] During training, the loss function of the Q-network is updated using the following formula:

[0066]

[0067] Where m is the current iteration number, T is the preset total number of iterations, and y m Let θ be the target value of the m-th objective Q-network. m Let be the loss function for the m-th iteration.

[0068] According to a second aspect of this application, a method for dynamically selecting the voltage control mode of a distribution network node is provided, comprising:

[0069] The voltage characteristics of each node in the distribution network at time t2 are collected. The voltage characteristics include: active power, reactive power, effective voltage value and voltage phase angle.

[0070] Based on the voltage characteristics of each node at time t2, an attention mechanism is used to calculate the weight of each node, and the weight calculation result is used as the attribute degree value to measure the relationship between the voltages of each node in the distribution network at time t2.

[0071] The attribute degree value is input into the first deep reinforcement learning model for training to obtain the occurrence frequency of each node when the model reaches the optimal target state. The first deep reinforcement learning model is the first deep reinforcement learning model built by the aforementioned model building method.

[0072] Based on the frequency of each node when the model reaches the optimal target state at multiple time points, create the frequency curve of each node at consecutive time points;

[0073] Based on the characteristics of the frequency curves corresponding to each node, the appropriate node voltage control mode is matched.

[0074] According to a third aspect of this application, a distribution network node voltage control system is provided, including a module for implementing the model establishment method for selecting the distribution network node voltage control mode as described above; or including a module for implementing the dynamic selection method for the distribution network node voltage control mode as described above.

[0075] According to a fourth aspect of this application, an electric power system is provided, including a distribution network node voltage control system as described above.

[0076] The dynamic selection method for distribution network node voltage control mode provided in this application directly uses the active power, reactive power, effective voltage value and voltage phase angle of each node in the distribution network as node features. It calculates the dynamic weights of the node features of the distribution network through a dynamic attention mechanism, and uses the calculated dynamic weights as the attribute degree value for node mode selection. Then, it designs the reward function of a deep reinforcement learning network based on the attribute degree value, and trains each node with the deep reinforcement learning network with the unique reward function. It calculates the occurrence frequency of each node when the model reaches the optimal target state, and then plots the occurrence frequency curve of each node at continuous time. Finally, it matches the corresponding node voltage control mode based on the characteristics of the occurrence frequency curve of each node.

[0077] This method comprehensively considers the active power, reactive power, RMS voltage, and voltage phase angle of each node as node voltage characteristics, improving the comprehensiveness and accuracy of node voltage control mode analysis compared to traditional single-consideration methods. It employs a dynamic attention mechanism to calculate the weights of each node's voltage characteristics and constructs a new training model based on the weight calculation results. This not only effectively, accurately, and comprehensively expresses the voltage change characteristics of each node but also comprehensively considers the interrelationships and mutual influence between these characteristics, achieving a comprehensive determination of the distribution network node voltage control mode. Furthermore, it utilizes deep reinforcement learning for dynamic analysis and decision-making regarding characteristic node voltage changes, fully leveraging... The algorithm demonstrates strong adaptability and generalization ability, effectively solving the complex decision-making problem of continuously changing voltage in distribution networks. It is the first to propose "node occurrence frequency" as a valuable variable for quantifying node voltage variation characteristics. This variable reflects the voltage variation levels of different nodes under different time periods and operating conditions, as well as the node voltage control potential and response capability, providing a reliable basis for selecting node voltage control modes. By using the node occurrence frequency curve at continuous time intervals as the basis for selecting node voltage control modes, the algorithm fully considers the dynamic performance of node characteristic variables, achieving the goal of selecting control modes of multiple types and time periods, and realizing the flexibility and adaptability of distribution network node voltage control.

[0078] In summary, this application can provide a scientific and reasonable auxiliary decision-making basis for the selection of voltage control modes at distribution network nodes, improve the effectiveness and accuracy of node voltage control mode selection, and thus enhance the efficiency and effectiveness of voltage control in the distribution network, ensuring the safe and stable operation of the power system.

[0079] Other features and advantages of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of what is pointed out in the written description and the accompanying drawings. Attached Figure Description

[0080] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0081] Figure 1 A flowchart illustrating a dynamic selection method for distribution network node voltage control modes according to an embodiment of this application;

[0082] Figure 2 A flowchart illustrating a model establishment method for selecting voltage control modes at distribution network nodes, provided in one embodiment of this application;

[0083] Figure 3 This is a frequency curve type diagram of a distribution network node provided in one embodiment of this application. Detailed Implementation

[0084] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0085] Addressing some problems existing in the current technology:

[0086] Firstly, this application provides a method for dynamically selecting the voltage control mode of a distribution network node. This method can be executed by a dynamic selection device for the voltage control mode of a distribution network node, or by components configured within the dynamic selection device, such as chips or chip systems. It can also be implemented by logic modules or software that have some or all of the functions of the dynamic selection device for the voltage control mode of a distribution network node. This application does not limit this method.

[0087] For example, such as Figure 1 As shown, the dynamic selection method for the voltage control mode of this distribution network node includes:

[0088] S101. Collect the voltage characteristics of each node in the distribution network at time t2. The voltage characteristics include: active power, reactive power, effective voltage value and voltage phase angle.

[0089] S102. Based on the voltage characteristics of each node at time t2, an attention mechanism is used to calculate the weight of each node, and the weight calculation result is used as the attribute degree value to measure the relationship between the voltages of each node in the distribution network at time t2.

[0090] S103. Input the attribute degree value into the first deep reinforcement learning model for training to obtain the occurrence frequency of each node when the model reaches the optimal target state. The first deep reinforcement learning model is not a conventional deep reinforcement learning model, but a deep reinforcement learning model built by a model building method specifically provided in this application embodiment. The specific building process is described in detail later.

[0091] S104. Based on the frequency of each node when the model reaches the optimal target state at multiple time points, create the frequency curve of each node at consecutive time points.

[0092] S105. Match the corresponding node voltage control mode according to the characteristics of the occurrence frequency curve of each node.

[0093] In some possible implementations of the first aspect, the weighting of each node based on the voltage characteristics of each node at time t2 using an attention mechanism, and the use of the weighting result as an attribute value for measuring the relationship between the voltages of each node in the distribution network at time t2, specifically includes:

[0094] Based on the active power, reactive power, effective voltage value and voltage phase angle of each node in the distribution network at time t2, feature matrices are established to represent the voltage change characteristics of each node in the distribution network at time t2.

[0095] Based on each feature matrix, construct the node feature matrix of the distribution network at time t2;

[0096] An attention mechanism is used to calculate the similarity between the features of each node in the node feature matrix, and the attention mechanism matrix is ​​obtained based on the similarity calculation results.

[0097] After normalizing the node feature matrix, the attention weight matrix is ​​calculated based on the attention mechanism matrix and the normalized node feature matrix.

[0098] The attention weight matrix is ​​used as the attribute degree value to measure the relationship between the voltages of each node in the distribution network at time t2.

[0099] As a crucial step in the embodiments of this application, the establishment of the first deep reinforcement learning model plays a key and decisive role in the rationality, accuracy, and effectiveness of the entire node voltage control mode selection result. Therefore:

[0100] Secondly, this application provides a model building method for selecting voltage control modes at distribution network nodes. This method can be executed by a model building device for selecting voltage control modes at distribution network nodes, or by components such as chips or chip systems configured inside the model building device. It can also be implemented by logic modules or software that have some or all of the functions of the model building device for selecting voltage control modes at distribution network nodes. This application does not limit this method.

[0101] For example, such as Figure 2 As shown, the model establishment method for selecting the voltage control mode of a distribution network node includes:

[0102] S201. Collect the voltage characteristics of each node in the distribution network at time t1. The voltage characteristics include: active power, reactive power, effective voltage value and voltage phase angle.

[0103] S202. Based on the voltage characteristics of each node at time t1, an attention mechanism is used to calculate the weight of each node, and the weight calculation result is used as the attribute degree value to measure the relationship between the voltages of each node in the distribution network at time t1.

[0104] S203. Based on the attribute degree values, construct the state space, action space, and reward function of the deep reinforcement learning model;

[0105] S204. Train each state space variable in the state space one by one, so that the deep reinforcement learning model reaches the optimal target state during training, and output the occurrence frequency of each node when it reaches the optimal target state.

[0106] S205. The successfully trained deep reinforcement learning model is used as the first deep reinforcement learning model for the final selection of voltage control mode at distribution network nodes.

[0107] In some possible implementations of the second aspect, the weighting of each node based on the voltage characteristics of each node at time t1 using an attention mechanism, and the weighting result used as an attribute value to measure the relationship between the voltages of each node in the distribution network at time t1, specifically includes:

[0108] Based on the active power, reactive power, effective voltage value and voltage phase angle of each node in the distribution network at time t1, voltage feature matrices are established to represent the voltage change characteristics of each node in the distribution network at time t1.

[0109] Based on the voltage characteristic matrices, construct the node characteristic matrix of the distribution network at time t1;

[0110] An attention mechanism is used to calculate the similarity between the features of each node in the node feature matrix (i.e., the importance of one node feature relative to another node feature), and the attention mechanism matrix is ​​obtained based on the similarity calculation results.

[0111] After normalizing the node feature matrix (normalizing each element in the node feature matrix to generate feature data with a mean of 0 and a variance of 1), the attention weight matrix is ​​calculated based on the attention mechanism matrix and the normalized node feature matrix.

[0112] The attention weight matrix is ​​used as the attribute degree value to measure the relationship between the voltages of each node in the distribution network at time t1.

[0113] In some possible implementations of the second aspect, the construction of the state space, action space, and reward function of the deep reinforcement learning model based on the attribute degree value specifically includes:

[0114] Based on the attention weight matrix, a state space of the deep reinforcement learning model is constructed, and the variables in the state space correspond one-to-one with the weight elements in the attention weight matrix.

[0115] Based on the mutual transition actions between state variables in the state space, the action space of the deep reinforcement learning model is constructed, wherein each transition action is executed by an ε-greedy policy.

[0116] Based on the attention weight matrix, the reward function of the deep reinforcement learning model is constructed as follows: when training a certain state space variable, if the weight element between the current state space variable and other state space variables is 0, the reward function value is 0; if the weight element between the current state space variable and other state space variables is not 0, the reward function value is the sum of the weight element between them and the next state space variable.

[0117] In some possible implementations of the second aspect, the deep reinforcement learning model is a DQN network model, wherein training each state space variable in the state space one by one, so that the deep reinforcement learning model reaches the optimal target state during training, and outputs the occurrence frequency of each node when the optimal target state is reached, specifically includes:

[0118] S2041. Randomly select a state space variable from the state space as the initial state space variable, input it into the training Q network of the first deep reinforcement learning model for training, and generate the corresponding initial training Q value (i.e., value).

[0119] S2042. Execute the action space according to the ε-greedy policy, calculate the corresponding reward function value according to the reward function, and form an experience dataset from each action information to build an experience pool.

[0120] S2043. Randomly select at least one set of experience datasets from the experience pool and input them into the training Q network and target Q network of the first deep reinforcement learning model respectively, and calculate the training Q value of the training Q network and the target value of the target Q network respectively.

[0121] S2044. Determine whether the training Q-value has reached the target value. If it has, proceed to step S2045 to determine whether the model has reached the required number of iterations. If it has not reached the target value, reselect the initial state space variables and start a new round of training.

[0122] If the model reaches the required number of iterations, then step S2046 is executed to determine whether the model has performed target state calculation for each node; if the model has not reached the required number of iterations, then the action space is executed according to the ε-greedy policy, and the next state space variable is reselected for training until the required number of iterations is reached.

[0123] If the model has performed target state calculation for each node, then step S2047 is executed to count and output the frequency of each node when the model reaches the optimal target state; if there are nodes that have not performed target state calculation, then the initial state space variables are reselected for a new round of training to solve the common local optimum problem.

[0124] In some possible implementations of the second aspect, the voltage characteristic matrix is ​​represented as:

[0125]

[0126] Where, x i.t1 Let p be the voltage characteristic matrix of the i-th node at time t1. i Let q be the active power of the i-th node. i Let U be the reactive power of the i-th node. i Let be the effective voltage value of the i-th node. Let be the voltage phase angle of the i-th node;

[0127] The node feature matrix is ​​represented as follows:

[0128]

[0129] Among them, X t1 Let n1 be the node characteristic matrix of the distribution network at time t1, where n1 is the number of nodes in the distribution network.

[0130] The attention mechanism matrix is ​​represented as follows:

[0131]

[0132] Among them, e t1 Let e ​​be the attention mechanism matrix at time t1. t1 The elements e in i.j Used to represent the node feature matrix X t1 Chinese x i.t1 and x j.t1 The similarity (which can also be described as x) i.t1 Relative to x j.t1 Importance);

[0133] The element e i.j Represented as:

[0134]

[0135] Where ReLU is the activation function, a is the trainable parameter vector, W is the trainable parameter matrix used to perform linear transformation on the input features, || represents the concatenation of vectors, and k represents the number of node features contained in the node feature matrix, k=1,……,n1;

[0136] The normalized node feature matrix is ​​represented as follows: , The elements in Represented as:

[0137]

[0138] in, and The node feature matrix X is respectively t1 The mean and variance, 1 represents the variance control bias, and γ1 and β are the learnable parameters of the normalized computation layer in the dynamic convolutional neural network. This represents the Hadamard product (i.e., element-wise multiplication).

[0139] Variance control bias 1 is generally taken as a very small value. In practical applications, by controlling the variance of the data, the sensitivity of the model to outliers can be reduced, thereby improving the stability and prediction accuracy of the model. Variance is the normalized distribution amplitude of the data. If the distribution amplitude has a very small deviation due to individual outliers (extreme values), a control bias needs to be set for correction. Usually, this value is 0. It is set when there is a deviation. In engineering calculations, the specific setting needs to take into account the deviation after the outlier calculation. γ1 and β are the learnable network parameters of the dynamic convolutional neural network. They are mainly used for linear feature extraction of normalized data samples. The network parameters are randomly generated in the network initialization and updated according to the features of each element in the training to achieve feature extraction.

[0140] The attention weight matrix is ​​represented as follows:

[0141]

[0142] Among them, W B.t1 Here is the attention weight matrix at time t1. For the activation function, this embodiment of the application uses the sigmoid function. This is a similarity matrix. for The transpose of the matrix;

[0143] The sigmoid function is a classic activation function. Its main function is to nonlinearly map the input feature values ​​to the (0, 1) or (-1, 1) region to achieve feature compression. The sigmoid function has good continuity and smoothness. In this embodiment, the activation function is used, which has good applicability for feature calculation of power grid nodes at continuous time. At the same time, the good smoothness is conducive to the optimal calculation of model training.

[0144] The similarity matrix Represented as:

[0145] .

[0146] In some possible implementations of the second aspect, the state space is represented as s t1 State space s t1 The state variables in the matrix correspond one-to-one with the attention weight matrix W. B.t1 The weighted elements in the space; the action space is represented as a. t1 In this embodiment of the application, state transition is used as the action space, a t1 The physical meaning is represented as the continuous action from the target state of node i to the target state of the next node j under the action of the ε-greedy strategy;

[0147] The reward function is expressed as follows:

[0148]

[0149] Where, r i The reward function corresponding to training the i-th node is the weight element w between the i-th node and the j-th node. ij When the value is 0, it indicates that there is no mutual influence between the voltage changes of the two nodes, and the reward function value is 0. When the weight element w between the i-th node and the j-th node is 0... ij When the value is not 0, it indicates that there is a mutual influence between the voltage changes of the two nodes, and the reward function value is the weight element w. ij With the next state space variable s j In this application embodiment, to guide the reinforcement learning model's state target towards the optimal target during reward function design, a next state value, s, is added to the reward function. j , which means that after the i-th node is taken as the target node, the model will select the state value of the next node according to the ε-greedy strategy, which is the node weight.

[0150] Based on the attention weight matrix W B.t1 The design of the reward function is a significant innovation of this application. Traditional graph networks often use adjacency matrices as input features for artificial intelligence models. However, in voltage control, nodes that are not directly connected may exhibit correlations during voltage changes. When using an adjacency matrix, the relationship between physically unconnected nodes is simply represented as "0," ignoring the correlation between electrical quantities. Therefore, this application directly uses an attention weight matrix to design the reward function, effectively reflecting the mutual influence of node voltage changes and providing a comprehensive, scientific, and reliable basis for the selection of the final control mode.

[0151] The ε-greedy strategy is expressed as follows:

[0152]

[0153] Wherein, π(a t1 |s t1 ) is the ε-greedy policy function; ε2 is the random selection probability, which takes the value ε2=max(0.01, 1-n2 / N), where n2 is the current training iteration and N is the total training iteration; argmaxQ(a t1 ,s t1Let ε be the maximum Q value corresponding to the optimal action; this strategy means that when the action is the optimal choice, its selection probability is 1-ε2, otherwise the selection probability is ε2. In the early stage of model training, this embodiment sets ε2 to a numerical value close to 1-n2 / N to represent the probability, so that the model can fully calculate the optimal target state of the node. As the number of training times increases, it ensures that the optimal target state is selected with the highest probability, thus accelerating the calculation speed.

[0154] In some possible implementations of the second aspect, the empirical dataset is represented as:

[0155] (s) i a t1 r i s j )

[0156] Among them, s i Let a be the current state space variable being trained. t1 For the currently executing action, r i For the corresponding reward function value, s j The next state-space variable is selected according to the state transition policy (i.e., the ε-greedy policy);

[0157] When d sets of empirical datasets are selected from the experience pool and input into the training Q-network and target Q-network of the first deep reinforcement learning model respectively, d <d m , where d m The data capacity of the experience pool;

[0158] During training, the following formula is used to calculate the target Q-network:

[0159]

[0160] Where m is the current iteration number, T is the preset total number of iterations, and y m Let T be the target value of the Q network for the m-th training iteration. When m reaches T, the instantaneous reward r obtained by training the network Q is... m The reward value; when m has not reached T, the long-term cumulative reward value (instant reward r) is used. m and the optimal Q value calculated in each iteration The sum of ( ) is the reward value; γ2 is the discount factor, and γ2 is introduced to limit the reward value to avoid it becoming infinitely large;

[0161] During training, the loss function of the Q-network is updated using the following formula:

[0162]

[0163] Where m is the current iteration number, T is the preset total number of iterations, and ym Let θ be the target value of the m-th objective Q-network. m Let θ be the loss function for the m-th iteration. In this embodiment, the backpropagation algorithm of the neural network is used to recalculate the θ value after updating the loss function, so that the Q value calculated by the trained Q network is closer to the target value calculated by the target Q network.

[0164] It should be noted that the time t2 mentioned in the mode selection method in the first aspect and the time t1 mentioned in the model building method in the second aspect both refer to "a certain moment" during the operation of the distribution network node. In order to prevent confusion in description and reference, this embodiment of the application uses the forms t1 and t2 to label a certain moment involved in the two different processes to clearly distinguish between the mode selection process and the model building process. In practical applications, t1 and t2 can be the same moment or different moments, and no limitation is made here.

[0165] In practical applications, this system can collect data from power grid nodes based on the power grid dispatch SCADA system. Attention weight matrix calculation and training of the first deep reinforcement learning model are performed every 15 minutes to obtain the node occurrence frequency at each time point and plot the node occurrence frequency curve. The node frequency at time t2, obtained from the first deep reinforcement learning model, can be expressed as: L t2 (1) L t2 (2), ..., L t2 (n); the frequency of occurrence of node i at consecutive times can be represented as: L1(i), L2(i), ..., L t (i), L t+1 (i), ..., L h (i), a total of h time points.

[0166] Once the node occurrence frequency curve is plotted, the node voltage control mode is selected based on the curve's characteristic trends. In practical applications, the node voltage control mode can correspond to six curve types, such as... Figure 3 As shown:

[0167] Curve ①: The frequency of node occurrence fluctuates within a high frequency range, indicating that this type of node always plays a significant role in the overall voltage control of the distribution network and has great adjustment potential, making it a better node for centralized control mode.

[0168] Curve ②: The frequency curve of the node continues to fluctuate in a low range, indicating that this type of node plays a very small role in the overall voltage control of the distribution network and has a weak regulation capability, and is usually not controlled.

[0169] Curve ③: The frequency of node occurrence gradually increases with the change of power grid operation time, indicating that this type of node can play a significant role in the overall voltage control of the distribution network under most power grid operating conditions, and has strong regulation capability. It can be selected as a centralized control mode for control according to the operating conditions.

[0170] Curve ④: The frequency of node occurrence gradually decreases with the change of power grid operation time, indicating that this type of node does not play a significant role in the overall voltage control of the distribution network under most power grid operating conditions, and its regulation capability is weak. It can be selected as a local control mode for control according to the operating conditions.

[0171] Curve ⑤: The frequency of node occurrence gradually decreases with the operation time of the power grid, and then gradually increases again at a certain moment. This indicates that this type of node can play a significant role in the overall voltage control of the distribution network under certain power grid operating conditions, and has strong regulation capabilities. It is generally controlled as a local control mode, and centralized control can be selected when the operating conditions are met.

[0172] Curve ⑥: The frequency of node occurrence gradually increases with the operation time of the power grid, and then gradually decreases at a certain moment. This indicates that this type of node can play a significant role in the overall voltage control of the distribution network under certain power grid operating conditions, and has strong regulation capabilities. It is generally controlled as a local control mode, and centralized control can be selected when the operating conditions are met.

[0173] Based on the above solution, the method provided in the embodiments of this application is as follows:

[0174] 1. By comprehensively considering the active power, reactive power, effective voltage value, and voltage phase angle of each node as node voltage characteristics, the comprehensiveness and accuracy of node voltage control mode analysis are improved compared with the traditional single consideration method.

[0175] 2. A dynamic attention mechanism is used to calculate the weights of the voltage characteristics of each node and a new training model is constructed based on the weight calculation results. This not only effectively, accurately and comprehensively expresses the voltage change characteristics of each node, but also comprehensively considers the interrelationships and mutual influence between the voltage characteristics of each node, thus realizing the comprehensive determination of the voltage control mode of the distribution network nodes.

[0176] 3. Deep reinforcement learning is used to analyze and make decisions on the voltage changes of feature nodes under dynamic time points, which gives full play to the strong adaptability and generalization ability of the algorithm and solves the complex decision-making problem of continuous voltage changes in the distribution network.

[0177] 4. For the first time, the "node occurrence frequency" is proposed as a good characterization variable for quantifying the characteristics of node voltage changes. It can reflect the voltage change level of different nodes under different time periods and operating conditions, and can also reflect the node voltage control potential and response capability, providing a reliable basis for the selection of node voltage control mode.

[0178] 5. The node occurrence frequency curve at continuous time intervals is used as the basis for selecting the node voltage control mode. The dynamic performance of the node characteristic variables is fully considered, so as to achieve the purpose of selecting multiple types and multiple time periods of control mode and realize the flexibility and adaptability of distribution network node voltage control.

[0179] Thirdly, this application provides a distribution network node voltage control system. Exemplarily, the distribution network node voltage control system includes a module for implementing the model establishment method for selecting the distribution network node voltage control mode as described above; or includes a module for implementing the dynamic selection method for the distribution network node voltage control mode as described above.

[0180] Fourthly, this application provides a power system, which, exemplarily, includes the distribution network node voltage control system as described above.

[0181] Fifthly, this application provides a device that can be any device capable of implementing the model establishment method for selecting voltage control modes of distribution network nodes as described above, or capable of implementing the dynamic selection method for voltage control modes of distribution network nodes as described above. The device can be various terminal devices, such as desktop computers, laptops, tablets, handheld devices, etc., and can be implemented through software and / or hardware.

[0182] For example, the device includes:

[0183] Memory;

[0184] Processor; and

[0185] Computer programs;

[0186] The computer program is stored in the memory and configured to be executed by the processor to implement the model building method for selecting voltage control modes of distribution network nodes as described above, or configured to be executed by the processor to implement the dynamic selection method for voltage control modes of distribution network nodes as described above.

[0187] Sixthly, this application provides a computer-readable storage medium, which may be a ROM, RAM, disk, or optical disk, etc.

[0188] For example, the computer-readable storage medium stores a computer program; the computer program is executed by a processor to implement the model building method for selecting voltage control modes of distribution network nodes as described above, or to implement the dynamic selection method for voltage control modes of distribution network nodes as described above.

[0189] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as C, VHDL, Verilog, the object-oriented programming language Java, and the interpreted scripting language JavaScript.

[0190] This application is described with reference to flowchart illustrations of methods, systems, and computer program products according to embodiments of this application. It should be understood that each step in the flowchart, and combinations of steps in the flowchart, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the steps in the flowchart. Figure 1 A device for a function specified in one or more processes.

[0191] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 The function specified in one or more processes.

[0192] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 Steps of a specified function in one or more processes.

[0193] In the description of this application, it should be understood that terms such as "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with terms such as "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0194] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0195] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A model establishment method for selecting voltage control modes at distribution network nodes, characterized in that: The method comprises the following steps: Collecting voltage characteristics of each node of the power distribution network at t1, wherein the voltage characteristics include active power, reactive power, voltage effective value and voltage phase angle; Based on the voltage characteristics of each node at t1, the weight of each node is calculated using an attention mechanism, and the weight calculation result is used as an attribute degree value for measuring the mutual relationship between the voltages of each node of the power distribution network at t1; According to the attribute degree value, the state space, action space and reward function of the deep reinforcement learning model are constructed; Each state space variable in the state space is trained one by one, so that the deep reinforcement learning model reaches the optimal target state in training and outputs the frequency of each node when it reaches the optimal target state; The trained deep reinforcement learning model is used as the final first deep reinforcement learning model for power distribution network node voltage control mode selection; Based on the voltage characteristics of each node at t1, the weight of each node is calculated using an attention mechanism, and the weight calculation result is used as an attribute degree value for measuring the mutual relationship between the voltages of each node of the power distribution network at t1, which specifically comprises: According to the active power, reactive power, voltage effective value and voltage phase angle of each node of the power distribution network at t1, a voltage characteristic matrix is established to represent the voltage variation characteristics of each node of the power distribution network at t1; According to each voltage characteristic matrix, a node characteristic matrix of the power distribution network at t1 is constructed; The similarity between each node characteristic in the node characteristic matrix is calculated using an attention mechanism, and an attention mechanism matrix is obtained according to the similarity calculation result; After normalizing the node characteristic matrix, the attention weight matrix is calculated according to the attention mechanism matrix and the normalized node characteristic matrix; The attention weight matrix is used as an attribute degree value for measuring the mutual relationship between the voltages of each node of the power distribution network at t1; According to the attention weight matrix, the state space of the deep reinforcement learning model is constructed, and the variables in the state space one-to-one correspond to the weight elements in the attention weight matrix; According to the mutual transition actions between each state variable in the state space, the action space of the deep reinforcement learning model is constructed, wherein each transition action is executed by an ε-greedy strategy; According to the attention weight matrix, the reward function of the deep reinforcement learning model is constructed: when training a certain state space variable, if the weight element between the current state space variable and other state space variables is 0, the reward function value is 0; if the weight element between the current state space variable and other state space variables is not 0, the reward function value is the sum of the weight element and the next state space variable. The deep reinforcement learning model is a DQN network model, and the training of each state space variable in the state space one by one, so that the deep reinforcement learning model reaches the optimal target state in training and outputs the frequency of each node when it reaches the optimal target state, specifically comprises:

2. The method for model establishment of voltage control mode selection of power distribution network nodes according to claim 1, characterized in that: ​ An initial state space variable is selected from the state space, and is input into a training Q network of the first deep reinforcement learning model for training, to generate a corresponding initial training Q value; An action space is executed according to an ε-greedy strategy, a corresponding reward function value is calculated according to a reward function, and each time of action information is formed into an experience dataset to build an experience pool; At least one set of experience datasets is selected from the experience pool, and is input into the training Q network and the target Q network of the first deep reinforcement learning model respectively, to calculate a corresponding training Q value of the training Q network and a target value of the target Q network respectively; It is judged whether the training Q value reaches the target value, if the target value is reached, it is judged whether the model reaches an iteration number, if the target value is not reached, an initial state space variable is reselected for a new round of training; If the model reaches the iteration number, it is judged whether the model has calculated the target state for each node, if the model does not reach the iteration number, a next state space variable is reselected for training according to the ε-greedy strategy, until the iteration number is reached; If the model has calculated the target state for each node, the frequency of each node when the model reaches the optimal target state is counted and output, if there is a node which has not calculated the target state, an initial state space variable is reselected for a new round of training.

3. The model establishment method for distribution network node voltage control mode selection according to claim 2, characterized in that: The voltage feature matrix is represented as: ； wherein x i.t1 is the voltage characteristic matrix of the i-th node at the time t1, p i is the active power of the i-th node, q i is the reactive power of the i-th node, U i is the voltage effective value of the i-th node, is the voltage phase angle of the i-th node; The node feature matrix is represented as: ； wherein X t1 is a node feature matrix of the power distribution network at time t1, and n1 is the number of nodes of the power distribution network. The attention mechanism matrix is represented as: ； wherein e t1 is the attention mechanism matrix at the time t1, e t1 is the element in e i.j is used to represent the similarity between x t1 and x i.t1 and x j.t1 in X The element e i.j is represented as: ； Wherein, ReLU is an activation function, a is a trainable parameter vector, W is a trainable parameter matrix, used for linear transformation of input features, || represents vector splicing, k represents the number of node features contained in the node feature matrix, k = 1, …, n1; The normalized node feature matrix is represented as , each element in the matrix is represented as: ； wherein, and are the mean and variance of the node feature matrix X t1 1 is a variance control bias, γ1 and β are learnable parameters of the dynamic convolutional neural network normalization computation layer, denotes the Hadamard product;​ The attention weight matrix is represented as: ； wherein W B.t1 is the attention weight matrix at time t1, is an activation function, is a similarity matrix, is is the transpose matrix of The similarity matrix is expressed as: 。 4. The model establishment method for distribution network node voltage control mode selection according to claim 3, characterized in that: The state space is denoted as s t1 , and the action space is denoted as a t1 ; The reward function is represented as: ； wherein r i is the reward function corresponding to the training of the i-th node, when the weight element w ij between the i-th node and the j-th node is 0, the reward function value is 0, when the weight element w ij between the i-th node and the j-th node is not 0, the reward function value is the sum of the weight element w ij and the next state space variable s j . The ε-greedy strategy is represented as: ； wherein, π(a t1 |s t1 ) is an ε-greedy policy function; ε2is a random selection probability, and is equal to ε2= max(0.01, 1-n2 / N), wherein n2is the current training number of the model, and N is the total training number; argmaxQ(a t1 ,s t1 ) is the maximum Q value corresponding to the optimal action; when the action is the optimal selection, the selection probability is 1-ε2, otherwise the selection probability is ε2.

5. The model establishment method for distribution network node voltage control mode selection according to claim 4, characterized in that: The experience dataset is represented as: (s i , a t1 , r i , s j ); wherein s i is the current state space variable being trained, a t1 is the action currently performed, r i is the corresponding reward function value, s j is the next state space variable selected according to the state transition policy; When d sets of experience data are selected from the experience pool and input into the training Q network and the target Q network of the first deep reinforcement learning model respectively, d < d m , where d m is the data capacity of the experience pool; In the training process, the following formula is used to calculate the target Q network: ； wherein m is the current iteration number, T is the total number of iterations set in advance, y m is the target value of the mth target Q network, and when m reaches T, the immediate reward r m obtained by training the network Q is the long-term cumulative reward value; γ2 is a discount factor; In the training process, the following formula is used to update the loss function of the training Q network: ； wherein m is the current iteration number, T is the total number of iterations set in advance, y m is the target value of the mth target Q network, θ m is the loss function of the mth iteration.

6. A method for dynamic selection of voltage control mode of a power distribution network node, characterized in that: Including: The voltage features of each node of the distribution network at t2 are collected, including active power, reactive power, voltage effective value and voltage phase angle; Based on the voltage features of each node at t2, the attention mechanism is used to calculate the weight of each node, and the weight calculation result is used as an attribute value for measuring the mutual relationship between the voltages of each node of the distribution network at t2; inputting the attribute value into a first deep reinforcement learning model for training to obtain an appearance frequency of each node when the model reaches an optimal target state, wherein the first deep reinforcement learning model is a first deep reinforcement learning model established by the model establishment method in any one of claims 1-5; creating an appearance frequency curve of each node at consecutive time instants according to the appearance frequency of each node at the model reaching the optimal target state at the plurality of time instants; matching a corresponding node voltage control mode according to a feature of the appearance frequency curve corresponding to each node.

7. A power distribution grid node voltage control system, characterized by: The model establishment method for node voltage control mode selection of a power distribution network in any one of claims 1-5 or the dynamic selection method of a node voltage control mode of a power distribution network in claim 6.

8. A power system, characterized by: The node voltage control system of a power distribution network in claim 7. The node voltage control system of a power distribution network in claim 7.

Images