Transformer multi-objective collaborative optimization control method, computer device and storage medium

Abstract

This invention discloses a multi-objective collaborative optimization control method for distribution transformer areas, a computer device, and a storage medium. The multi-objective collaborative optimization control method for distribution transformer areas includes the following steps: During model training, distribution network measurement data, the distribution network adjacency matrix, and control target values ​​are acquired; a graph structure input vector matrix is ​​constructed; and the GCN-Critic model is trained. This yields a multi-objective voltage reactive power model and an active power command model capable of automatically generating control commands based on real-time grid conditions. The voltage reactive power model and active power command model include a GCN strategy network and a Critic evaluation network. During online control, based on real-time measurement data, a graph structure input vector matrix is ​​constructed for the current time period; the trained voltage reactive power model / active power command model is input; and the GCN strategy network outputs the optimal control commands for controllable nodes. This invention only requires the GCN decision network to acquire actual measurements of the characteristics of each node on the line, and can output the optimal control commands for each node in real time.

Description

Technical Field

[0001] This invention relates to the field of distribution network automation and voltage control technology, and in particular to a multi-objective collaborative optimization control method for distribution transformer areas. Specifically, it relates to a multi-objective collaborative optimization control method for distribution transformer areas that integrates graph convolutional networks and SAC framework, as well as computer equipment and storage media. Background Technology

[0002] Power system optimization dispatch, a key technology for ensuring the safe and economical operation of power systems, has core value in maintaining voltage stability, reducing network losses, and improving equipment utilization. With the growth of electricity demand and the expansion of the power grid, the increasing complexity of the system makes traditional methods insufficient to meet the demands of modern power grids for real-time response and flexible dispatch. Distribution networks with high penetration rates of renewable energy sources are more prone to problems such as local line overload and localized overvoltage due to the randomness, volatility, and intermittency of renewable energy sources. The large-scale integration of power electronic equipment and the grid connection of new loads such as distributed energy storage pose new challenges to harmonic mitigation, grid loss reduction, and voltage control, and also place higher demands on the optimization control capabilities of the distribution network, urgently requiring the development of new optimization strategies. Simultaneously, to achieve coordinated control of medium- and low-voltage distribution networks, new control requirements are being placed on low-voltage distribution areas. The voltage or active power at the connection point with the upstream medium-voltage feeder must be controlled within the target values ​​set by the upstream dispatching department according to the medium-voltage distribution network control specifications, in order to achieve flexible control of the voltage and active power of low-voltage distribution areas. Therefore, it is necessary to provide a new multi-objective collaborative optimization control method for distribution areas to solve the above-mentioned technical problems. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings and defects of the prior art and to provide a multi-objective collaborative optimization control method, computer equipment and storage medium for transformer substations that integrates graph convolutional networks and SAC framework.

[0004] One objective of this invention is to provide a multi-objective collaborative optimization control method for transformer substations, comprising the following steps:

[0005] During model training, distribution network measurement data, distribution network adjacency matrix, and control target values ​​are acquired. After data preprocessing, a graph structure input vector matrix is ​​constructed, and the constructed GCN-Critic model structure is trained iteratively. Depending on the control target values, the final multi-objective control voltage reactive power model and active power command model, which can automatically generate control commands based on real-time grid conditions, are obtained. The voltage reactive power model and active power command model include a GCN strategy network and a Critic evaluation network. The GCN strategy network receives the graph structure input vector matrix and outputs the control action commands for the distribution areas. The Critic evaluation network outputs scores to quantitatively evaluate the multi-objective control performance after executing control action commands under the current grid conditions.

[0006] During online control, the system acquires the current distribution network measurement data and control target value to determine the current control mode. Based on the current control mode, it selects the voltage reactive power model / active power command model. Using the real-time acquired distribution network measurement data, real-time control target value, and preset distribution network adjacency matrix, it constructs the graph structure input vector matrix for the current moment. The trained voltage reactive power model / active power command model is then input, and the GCN policy network performs inference calculations based on the learned mapping relationship, outputting the optimal control command set for controllable nodes to achieve real-time optimized control of the transformer area.

[0007] Preferably, during model training, the control target value includes the voltage target value or active power target value of the upstream medium-voltage feeder connection node.

[0008] Preferably, during online control, the current control mode is determined based on the real-time control target value. If the real-time control target value is the target value for tracking the voltage of the upstream feeder, then the voltage reactive power model is loaded and enabled; if the real-time control target value is the target value for tracking the active power of the upstream feeder, then the active power command model is loaded and enabled.

[0009] Preferably, during online control, after the step of optimizing the control of the transformer area by outputting the optimal control instruction set for controllable nodes, the method further includes:

[0010] The system continuously monitors and acquires operational data after regulation and feeds it back to the data center. Newly generated operational data is stored and used to periodically perform incremental training or fine-tuning of the GCN-Critic model, enabling the GCN-Critic model to adapt to changes in the power grid structure and the evolution of operational characteristics, thus achieving self-evolution. When new measurement data arrives, the time corresponding to the new measurement data is used as the new "current time," and the aforementioned steps are repeated to achieve real-time optimization and regulation of the transformer substation.

[0011] Preferably, during model training, a graph-structured input vector matrix is ​​constructed, including:

[0012] Construct the quadruplet input vector data:

[0013] ,

[0014] in, The input vector matrix represents the graph structure. For a set of nodes, corresponding to the distribution network One physical node; Let the set of edges represent electrical connections; The node feature matrix integrates multi-dimensional operational status information. This is the adjacency matrix of the distribution network, describing the network topology.

[0015] Each node Corresponding to a feature vector ,in For feature dimensions;

[0016] ;

[0017] in, Injecting active power into the node; Inject reactive power into the node; The node voltage amplitude is the direct indicator of voltage quality. Energy state of the energy storage system; This is a connection identifier for the upstream feeder; This is the target value set by the higher authorities.

[0018] For the upstream medium-voltage feeder connection node, mark it as ;

[0019] , ;

[0020] in, The target value for voltage regulation at the connection point of the medium-voltage feeder; The target value for active power regulation at the connection point of the medium-voltage feeder; , These represent the target voltage and target power values ​​at the connection node of the upstream medium-voltage feeder, respectively.

[0021] For non-connected nodes , ;

[0022] The node feature matrix is ​​constructed based on the node order as follows:

[0023] ;

[0024] Distribution network adjacency matrix Based on the physical topology of the distribution network, its elements Defined as:

[0025]

[0026] Among them, nodes With nodes The value is 1 for direct electrical connection and 0 for other cases.

[0027] Preferably, during model training, the GCN policy network receives a graph structure input vector matrix and outputs substation control action commands, including:

[0028] The first graph convolutional layer is used based on the distribution network adjacency matrix. and node feature matrix Local feature aggregation is achieved, and one-hop neighborhood aggregation is performed; the second graph convolutional layer is used to perform aggregation based on the distribution network adjacency matrix. and the first layer output features Global feature extraction is achieved, and the receptive field is expanded to a two-hop neighborhood to obtain the second layer of output features. ;include:

[0029] ; ;

[0030] , This represents the weight matrices for the first and second graph convolutional layers. for Activation function; For standardized distribution network adjacency matrix;

[0031] Second layer output features Compressed into a single global feature vector using global average pooling. ;

[0032] ,in, To represent the total number of nodes in the distribution network;

[0033] Single global feature vector After nonlinear transformation by a fully connected layer, the Gaussian distribution parameters of the GCN policy network are output based on the transformed vector, including the mean vector and standard deviation vector of the actions. Based on these Gaussian distribution parameters, the policy of the GCN policy network is determined. According to the strategy Unbounded action samples are obtained by sampling the commands and actions issued by the device. The hyperbolic tangent function is used to transform unbounded action samples. Compressing to the [-1,1] interval yields the control command range. ;

[0034] Wherein, the vector after nonlinear transformation is , This is the weight matrix of the fully connected layer. For bias terms of fully connected layers;

[0035] Gaussian distribution parameters include , ; Obtained by the GCN policy network through mapping the power grid state s, This is the mean vector of the actions; The standard deviation vector of the action; The bias vector for the mean output layer; The standard deviation is the output layer bias vector; This represents the weight matrix of the mean output layer. The standard deviation represents the output layer weight matrix;

[0036] Based on Gaussian distribution parameters, determine the policy of the GCN policy network. as follows:

[0037] ;

[0038] The sampling of command actions is performed using the following formula:

[0039] ;

[0040] Where ⊙ represents element-wise multiplication; It is standard normal random noise.

[0041] Preferably, during model training, the reward function of the voltage reactive power model is... From voltage error term Network loss minimization item Root node voltage tracking term The result is obtained by weighting the sum according to different weighting coefficients:

[0042] ;

[0043] in, , and These are the weighting coefficients, weighting coefficients , and Adjustments are made based on control priority, with a total of 1.

[0044] Reward function of voltage reactive power model middle,

[0045] Voltage error term ;

[0046] Network loss minimization term ;

[0047] Root node voltage tracking term ;

[0048] in, Indicates time node The voltage; This represents the set of non-root nodes, that is, the set of all nodes excluding those connected to the upper-level medium-voltage feeder. Represents the voltage-reactive power model Real-time active power loss of the entire network. Indicates time branch road The line current; Indicates the branch resistance; Represents the set of branches; This refers to the root node voltage amplitude connected to the upstream medium-voltage feeder; Set the target voltage value for the root node; To track the intensity coefficient.

[0049] Preferably, during model training, the active instruction model reward function... From voltage error term Network loss minimization item Root node active power tracking item Controlling costs and penalties The result is obtained by weighted summation using different weighting coefficients:

[0050] ;

[0051] in, > > > ， The sum is 1;

[0052] Voltage error term ;

[0053] Network loss minimization term ;

[0054] Root node active power tracking term ;

[0055] Control cost penalty items , ;

[0056] in, Indicates time node The voltage; and These represent the upper and lower limits of the allowable voltage, respectively. Represents a set of nodes. Indicating active power instruction model Real-time active power loss of the entire network. Indicates time branch road The line current; Indicates the branch resistance; Represents the set of branches. Indicates time The active power of the root node connected to the upper-level medium-voltage feeder; The target value for active power control at the root node; To track the intensity coefficient, Represents a node At any moment The active power regulation amount, , Representing time respectively , Active power; Represents a node Adjustment weighting coefficient, weighting coefficient The nodes are categorized according to their importance, and sorted by priority: critical load nodes, ordinary load nodes, and adjustable resource nodes.

[0057] A second objective of this invention is to provide a computer device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-objective collaborative optimization control method for the distribution area.

[0058] A third objective of this invention is to provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the multi-objective collaborative optimization control method for the transformer substation.

[0059] In the model training phase, this invention uses historical data from measurements of active and reactive power injection, voltage amplitude, and the current energy level of the energy storage system at each node in the distribution area as a sample dataset. This dataset, along with the corresponding time series data, forms a sample matrix. This sample matrix is ​​then combined with the distribution network adjacency matrix and the voltage or active power target values ​​of the upstream medium-voltage feeder connection nodes, and converted into graph-structured data to obtain the input vector matrix. The policy network employs Graph Convolutional Networks (GCNs) to receive the input vector matrix and output control action commands. The Critic network evaluates the multi-objective control effect of these commands under the corresponding grid conditions. The policy network and the evaluation network are trained iteratively. Depending on whether the control objective of the medium-voltage feeder connection nodes is voltage or active power, high-precision voltage and reactive power models and active power command models for the distribution area are trained separately.

[0060] In the online control decision-making stage of this invention, the voltage reactive power model or active power command model that has been trained is selected according to different control objectives. Only the corresponding GCN decision network needs to obtain the actual measurement of the characteristics of each node of the line, and the optimal control command for each node can be output in real time.

[0061] Compared with existing methods for optimizing and controlling distribution areas, this invention provides a more promising technical path for constructing intelligent control strategies that do not require precise physical parameters and can adapt to changes in network structure in the context of highly distributed renewable energy access by replacing the Actor network in the conventional SAC (Soft Actor-Critic) model deep reinforcement learning with GCN as the policy network.

[0062] This invention integrates the SAC framework, performing iterative training on the policy and evaluation function during the training process, enabling the policy network to make better decisions. In multi-target coordination, the voltage or active power control target of the upper-level connected node is introduced to meet the key requirements of flexibility and controllability of voltage and active power regulation in low-voltage distribution areas under the medium and low voltage coordinated control framework. Attached Figure Description

[0063] Figure 1 This is a flowchart of the model training process for the multi-objective collaborative optimization control method for power distribution areas of this invention.

[0064] Figure 2 This is a flowchart of the online control decision-making process for the multi-objective collaborative optimization control method for power distribution areas according to the present invention.

[0065] Figure 3 This is a diagram of the GCN structure in the GCN-Critic model of this invention.

[0066] Figure 4 This is a structural diagram of the GCN-Critic model of the present invention.

[0067] Figure 5 This is a flowchart of the GCN-Critic model network parameter optimization and update process of the present invention.

[0068] Figure 6 This is a flowchart of the online application of the GCN-Critic model of the present invention. Detailed Implementation

[0069] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0070] See Figure 1 and Figure 2 As shown in the exemplary embodiment of this application, the multi-objective collaborative optimization control method for transformer substations includes a model training phase and an online decision-making phase:

[0071] See Figure 1As shown, during model training, distribution network measurement data, distribution network adjacency matrix, and control target values ​​are acquired. After data preprocessing, a graph structure input vector matrix is ​​constructed, and the constructed GCN-Critic model structure is trained iteratively. Depending on the control target values, the final voltage reactive power model and active power command model, capable of automatically generating control commands based on real-time grid conditions, are obtained through training. The voltage reactive power model and active power command model include a GCN strategy network and a Critic evaluation network. The GCN strategy network receives the graph structure input vector matrix and outputs control action commands for distribution areas. The Critic evaluation network outputs scores to quantitatively evaluate the multi-objective control performance after executing control action commands under the current grid conditions.

[0072] See Figure 2 As shown, during online control, the current distribution network measurement data and control target value are acquired to determine the current control mode. Based on the current control mode, the control model is selected: whether to activate the voltage reactive power model or the active power command model. Based on the real-time acquired distribution network measurement data, the real-time control target value, and the preset distribution network adjacency matrix, a graph structure input vector matrix for the current moment is constructed. The trained voltage reactive power model / active power command model is input, and the GCN policy network infers and calculates based on the learned mapping relationship, outputting the optimal control command set for controllable nodes, thereby realizing real-time optimized control of the transformer area.

[0073] In this embodiment of the application, during model training, the control target value includes the voltage target value or active power target value of the upstream medium-voltage feeder connection node. The distribution network measurement data includes at least historical data of active and reactive power injection, voltage amplitude, and current energy level of the energy storage system at each node in the distribution area. During training, this data serves as a sample dataset, which, together with the time series corresponding to the measurements, constitutes a sample matrix. This sample matrix is ​​then combined with the distribution network adjacency matrix and the voltage or active power target value of the upstream medium-voltage feeder connection node, and converted into graph structure data to obtain the input vector matrix.

[0074] In this embodiment of the application, during online decision-making, the required control mode is first determined based on the real-time control target value, and the corresponding control model is selected. If the current real-time control target value is to track the upper-level feeder voltage target value, then the voltage reactive power model is loaded and enabled for online decision-making control; if the current real-time control target value is to track the upper-level feeder active power target value, then the active power command model is loaded and enabled for online decision-making control.

[0075] In the online decision-making and control process, real-time distribution network measurement data is obtained through sensors and measuring devices deployed at various nodes in the distribution area, and real-time control target values ​​issued by the superior dispatching department are obtained, such as the target values ​​of voltage or active power at the connection point of the superior medium-voltage feeder set by the superior dispatching department.

[0076] In this exemplary application, during the training of the GCN-Critic model, the construction of a graph-structured input vector matrix includes:

[0077] The input vector quadruples of the graph structure input vector matrix are represented as follows:

[0078] ,

[0079] in, The input vector matrix represents the graph structure. This is a set of nodes, corresponding to each physical node in the distribution network; Let the set of edges represent electrical connections; The node feature matrix integrates multi-dimensional operational status information. It is a distribution network adjacency matrix that can accurately describe the network topology;

[0080] Each node Corresponding to a feature vector ,in For feature dimensions;

[0081] ;

[0082] in, Injecting active power into the node; It is used for reactive power injection at nodes and as a key voltage regulation variable. The node voltage amplitude is the direct indicator of voltage quality. To determine the energy status of the energy storage system and ensure control feasibility; This is the identifier for the upstream feeder connection (binary variable). This is a target value set by the higher-level regulator; it is only valid for the higher-level connected nodes.

[0083] For the upstream medium-voltage feeder connection node, mark it as ;

[0084] , ;

[0085] in, The target value for voltage regulation at the connection point of the medium-voltage feeder; The target value for active power regulation at the connection point of the medium-voltage feeder; , These represent the target voltage and target power values ​​at the connection node of the upstream medium-voltage feeder, respectively.

[0086] For non-connected nodes , ;

[0087] The feature matrix is ​​constructed based on the node order as follows:

[0088]

[0089] Distribution network adjacency matrix Based on the physical topology of the distribution network, its elements Defined as:

[0090] ;

[0091] Among them, nodes With nodes The value is 1 for direct electrical connection and 0 for other cases.

[0092] After constructing the input vector matrix through the above steps, it is also necessary to preprocess the input vector matrix and feature label matrix. By preprocessing the original data, it can meet the needs of model training and improve the accuracy of model prediction.

[0093] Specifically, the data preprocessing includes data denoising and data normalization.

[0094] Data denoising: When measurement devices in a power distribution network collect and transmit measurement data, both the devices themselves and the external environment can affect the data. Therefore, it is necessary to denoise the measurement data to restore the true data as much as possible and ensure the quality of the input data for the measurement repair model. For example, an approximately symmetric compactly supported orthogonal wavelet (Symlet) denoising function can be used for preliminary processing of the measurement data. Symlet wavelets have excellent symmetry, which can reduce distortion during data denoising.

[0095] Data normalization: Data normalization maps preprocessed data to a fixed range (e.g., [0,1] or [-1,1]) according to a certain mapping relationship. This can eliminate the impact of outlier data on model training performance, accelerate gradient descent, find the optimal solution faster, and improve model convergence speed and training accuracy. For example, the Min-Max normalization method maps data to the [-1,1] interval according to a certain mapping relationship, achieving data normalization. The mapping function is:

[0096] ;

[0097] in, =1, =-1, This is the original data. This is the normalized data.

[0098] The GCN policy network described in this application is a graph convolutional network structure, which consists of four parts: an input layer, a convolutional layer, a pooling layer, and an output layer.

[0099] For stable training of the model, the adjacency matrix of the distribution network needs to be optimized. Standardization is performed to obtain a standardized distribution network adjacency matrix. :

[0100] ;

[0101] Among them, is It is the adjacency matrix of the distribution network. It is a degree matrix. It is the identity matrix. This indicates the addition of a self-loop.

[0102] After one layer of propagation in GCN, the node features will be updated using the formula above:

[0103] ;

[0104] in, For standardized distribution network adjacency matrix; The node feature matrix; This is the weight matrix (learnable parameters) for the first layer. This is the activation function.

[0105] In a GCN policy network, a single-layer GCN focuses on capturing local topological relationships between nodes. By stacking multiple layers, global structural features of the graph can be extracted step by step, achieving multi-level feature extraction and fusion of graph structure data. Its layer-by-layer propagation rules are as follows:

[0106] ;

[0107] in, For the first The weight matrix of the layer (learnable parameters), the first layer Layer output It becomes the input of the next layer; the output of the last layer. It is typically used for classification or regression tasks.

[0108] Specifically, in this application, during the training of the GCN-Critic model, a two-layer graph convolutional layer (GCNlayer) structure is used to extract features from the input vector. The features are directly output by the second GCNlayer, and the output results are used for classification or regression tasks. See [link to relevant documentation]. Figure 3 As shown, the steps include:

[0109] The first graph convolutional layer is used based on the distribution network adjacency matrix. and node feature matrix Local feature aggregation is achieved, and one-hop neighborhood aggregation is performed; the second graph convolutional layer is used to perform aggregation based on the distribution network adjacency matrix. and the first layer output features Global feature extraction is achieved, and the receptive field is expanded to a two-hop neighborhood to obtain the second layer of output features. In a tree topology, two layers of GCN can cover the end-to-end range of a typical distribution area;

[0110] ; ;

[0111] , This represents the weight matrices for the first and second graph convolutional layers. for Activation function; For standardized distribution network adjacency matrix;

[0112] To generate system-level control decisions, global average pooling is used to combine the output features of the second layer. Compressed into a single global feature vector using global average pooling. ;

[0113] ,in, To represent the total number of nodes in the distribution network;

[0114] Single global feature vector After nonlinear transformation by the fully connected layer, the Gaussian distribution parameters of the GCN policy network, including the mean vector of the actions, are output based on the transformed vector. with the standard deviation vector of the action Determine the strategy based on Gaussian distribution parameters. ,Strategy Defined as For the mean, The variance is a Gaussian distribution; according to the strategy Sample the commands and actions issued by the control equipment to obtain unbounded action samples. The hyperbolic tangent function is used to transform unbounded action samples. Compressing to the [-1,1] interval yields the control command range. ;

[0115] Wherein, the vector after nonlinear transformation is , This is the weight matrix of the fully connected layer. For bias terms of fully connected layers;

[0116] Among them, the Gaussian distribution parameters include , ; Obtained by the GCN policy network through mapping the power grid state s, This is the mean vector of the actions; The standard deviation vector of the action; The bias vector for the mean output layer; The standard deviation is the output layer bias vector; This represents the weight matrix of the mean output layer. The standard deviation represents the output layer weight matrix;

[0117] Among them, strategy Represented as: ;

[0118] The sampling of commands and actions issued by the control equipment is achieved through reparameterization techniques:

[0119] Where ⊙ represents element-wise multiplication, It is standard normal random noise.

[0120] Original Gaussian sampling action sample Since the input is unbounded, the hyperbolic tangent function is used to compress the unbounded action samples to the interval [-1, 1] to meet the control command range of practical devices (such as PV inverters and SVCs).

[0121] ;

[0122] in, This is the final control action command for the GCN policy network.

[0123] Based on the above, the GCN policy network outputs control action commands for the station area.

[0124] In model training, to improve its ability to explore using the DRL (Reinforcement Learning) algorithm and avoid local minima, the model adopts a soft objective. Maximum entropy exploration encourages trying new actions, avoids premature convergence, and can systematically discover the globally optimal coordination strategy. The objective function of the GCN (Global Networking Community) policy network considers cumulative reward and entropy.

[0125] ;

[0126] in, This is the optimal strategy; To find a strategy that maximizes the expected value; State-action pair compliance policy The expected value under the distribution; Represents the entropy term; For instant rewards; This represents a temperature factor used to balance the weights between cumulative rewards and entropy. This represents the discount factor.

[0127] Critic evaluation network's comment function ( The value is an action value function, designed to evaluate the value of an action in a given state. Next action The quality of the network is evaluated using the Critic evaluation function, which introduces an entropy term based on the maximum entropy objective:

[0128] ;

[0129] in Indicates the state Next action The cumulative rewards obtained at that time.

[0130] The parameters of the Critic evaluation network are updated by minimizing the residuals, and the loss function is:

[0131] ;

[0132] in, This represents the loss function of the Critic evaluation network. This represents the experience replay pool in reinforcement learning. This means randomly selecting a batch of samples from the experience replay pool. Based on samples Calculate the expected value of the subsequent loss function.

[0133] The parameters of the GCN policy network are updated by maximizing the expected reward and entropy, and its loss function is:

[0134] ;

[0135] in, Let be the loss function of the GCN policy network; To sample the expected state from the experience replay buffer, Represents learnable parameters of function.

[0136] In this application, based on the different control targets of voltage or active power at the medium-voltage feeder connection point set by the superior dispatching department, a reward function is designed in a targeted manner, and a multi-objective control voltage reactive power model and an active power command control model are constructed and trained to obtain the final multi-objective control voltage reactive power model and active power command control model.

[0137] In this embodiment, based on minimizing the network loss and ensuring the node voltage is as close as possible to the rated voltage, it is further required that the voltage at the connection node between the transformer area and the upstream medium-voltage feeder be as close as possible to the target voltage value. Therefore, the reward function of the voltage reactive power model is designed. From voltage error term Network loss minimization item Root node voltage tracking term The result is obtained by weighting the sum according to different weighting coefficients:

[0138] ;

[0139] in, , and These are the weighting coefficients, weighting coefficients , and Adjustments are made based on control priority, with a total sum of 1. The weighting coefficients are adjusted according to the control priority.

[0140] For example, in this application, the reward function of the voltage reactive power model middle,

[0141] Voltage error term ;

[0142] Network loss minimization term ;

[0143] Root node voltage tracking term ;

[0144] in, Indicates time node The voltage; This represents the set of non-root nodes, that is, the set of all nodes excluding those connected to the upper-level medium-voltage feeder. Represents the voltage-reactive power model Real-time active power loss of the entire network. Indicates time branch road The line current; Indicates the branch resistance; Represents the set of branches; This refers to the root node voltage amplitude connected to the upstream medium-voltage feeder; Set the target voltage value for the root node; To track the intensity coefficient, the relative importance of the target is controlled.

[0145] In the active power command model, its reward function needs to simultaneously optimize four core objectives: voltage stability (ensuring all node voltages remain within their rated operating range); network loss minimization (reducing total system active power loss); active power tracking (ensuring the root node's active power accurately tracks the given target from the higher level); and control cost minimization (reducing node power regulation amplitude and prioritizing the protection of critical nodes). Therefore, the reward function of the active power command model needs to be constructed to comprehensively consider multiple objectives. For example, its model reward function... From voltage error term Network loss minimization item Root node active power tracking item Controlling costs and penalties The result is obtained by weighted summation using different weighting coefficients:

[0146] ;

[0147] in, > > > The sum is 1, that is Among these, the weighting coefficients need to be adjusted according to actual engineering needs, ensuring that voltage safety takes precedence over power point tracking, and power point tracking takes precedence over network loss optimization and cost control.

[0148] In this embodiment of the application, the active power instruction model reward function is exemplary. middle,

[0149] Voltage error term The voltage over-limit penalty item adopts an over-limit penalty mechanism to ensure that the voltage of each node is maintained within the allowable operating range;

[0150] Network loss minimization term Network minimization encourages reducing the total active power loss of the system.

[0151] Root node active power tracking term The root node active power tracking reward ensures that the root node's active power accurately tracks the target given by the superior.

[0152] Control cost penalty items , The cost control penalty is used to limit the power adjustment range of each node and to assign higher weights to important nodes.

[0153] in, Indicates time node The voltage; and These represent the upper and lower limits of the allowable voltage (usually...). =1.05 pu, =0.95 pu); This represents the set of nodes, specifically the set of all nodes excluding those connected to the upstream medium-voltage feeder. Indicating active power instruction model The total active power loss of the entire network at all times. Indicates time branch road The line current; Indicates the branch resistance; Represents the set of branches. Indicates time The active power of the root node connected to the upper-level medium-voltage feeder; The target value for active power control at the root node; To track the intensity coefficient and control the relative importance of the active power tracking target, Represents a node At any moment The active power regulation amount, , Representing time respectively , Active power; Represents a node Adjustment weighting coefficient, weighting coefficient The nodes are prioritized according to their importance, categorized as critical load nodes, ordinary load nodes, and adjustable resource nodes. For example, the weight coefficient for critical load nodes is [not specified]. Ordinary load node weighting coefficient Adjustable resource node weight coefficient And satisfy .

[0154] The preprocessed input vector matrix is ​​used as input data to train the GCN-Critic model parameters (weights and biases). The GCN policy network generates actions based on the current state, while the Critic evaluation network assesses the regulatory effect of these actions. Utilizing the state features and the value signals provided by the evaluation network, the parameters of both the policy network and the evaluation network are updated via backpropagation. Ultimately, this yields a voltage reactive power model and an active power command regulation model for the distribution transformers, capable of automatically generating precise regulation commands based on real-time grid conditions.

[0155] like Figure 4 , Figure 5 As shown, for example, the training method for the GCN-Critic model in this application is as follows: At the beginning of the training process, the GCN policy network obtains the power grid state through the power grid historical database. Compared with the target value of the higher-level regulation Based on the extracted features, the GCN policy function generates a control strategy and outputs the control actions for the control equipment. The current power grid state is obtained by calculating using a power flow calculation model. Execute control actions Post-system node voltage and power flow distribution, calculation of reward function Afterwards, the system reaches a new power grid state. Experience states are in the form of quintuples. It is stored in the experience playback buffer.

[0156] Batch experience state data is sampled from the experience replay buffer, and the comment function is used to calculate... The values ​​are then used to further calculate the losses of the GCN policy network and the Critic evaluation network, and to update the parameters of the policy network and the evaluation network.

[0157] Among them, the Critic evaluation network parameters In each update, the control strategy determined by the strategy network and the evaluation indicators calculated by the evaluation network are used. The network evaluation parameters are updated, and the network loss is as follows:

[0158] ;

[0159] ;

[0160] in, This indicates the learning rate used by the Critic to evaluate the network. The number of loop iterations. Represents model parameters Loss function value.

[0161] In this application, the parameters of the GCN policy network are described. Every two rounds of updates, the policy variance and policy mean are obtained through the policy network to determine the control policy. The network parameters are then updated based on the control policy, and the network loss function is as follows:

[0162] ;

[0163] ;

[0164] in, This represents the learning rate of the GCN policy network. Represents model parameters Loss function value.

[0165] In this application, the parameters of the target network, which consists of the GCN policy network and the Critic evaluation network, are updated using a soft update strategy to maintain training stability. The update method is as follows, resulting in the updated parameters. , ;

[0166] ;

[0167] Where τ<<1 is the soft update coefficient.

[0168] Through the above iterative training and network parameter updates, the model or network with updated network parameters is finally obtained. The structure, parameters, and results of the trained GCN-Critic model (optimized control model) are saved as the optimized control model of the trained GCN-Critic architecture.

[0169] Through the above training, a well-trained GCN-Critic architecture model is finally obtained, which yields a high-precision reactive power model and active power command model for transformer substation voltage, which can then be used for control decision-making.

[0170] During the online control decision-making phase, the trained voltage reactive power model or active power command model is selected based on the different control objectives. Only the corresponding GCN strategy network needs to obtain the actual measurement data of the characteristics of each node of the line, and then adjust the model according to the current state of the power grid. This allows for the real-time output of optimal control commands for each node, i.e., control actions. Then the operation is carried out by the station area, such as Figure 6 As shown.

[0171] Compared with existing methods for optimizing and controlling distribution areas, this invention provides a more promising technical path for constructing intelligent control strategies that do not require precise physical parameters and can adapt to changes in network structure in the context of highly distributed renewable energy access by replacing the Actor network in the conventional SAC (Soft Actor-Critic) model deep reinforcement learning with GCN as the policy network.

[0172] The technology in this application, by integrating the SAC framework, iteratively trains the policy and evaluation function during the training process, enabling the policy network to make better decisions. In multi-target coordination, the voltage or active power control target of the upper-level connected node is introduced to meet the key requirements of flexibility and controllability of voltage and active power regulation in low-voltage distribution areas under the medium- and low-voltage coordinated control framework.

[0173] In a further embodiment of this application, a computer device is provided, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the multi-objective collaborative optimization control method for the transformer area.

[0174] In a further embodiment of this application, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the multi-objective collaborative optimization control method for the transformer area.

[0175] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention.

[0176] Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of the equivalents of the claims be included within the invention.

[0177] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A multi-objective collaborative optimization control method for transformer substations, characterized in that, Includes the following steps: During model training, distribution network measurement data, distribution network adjacency matrix, and control target values ​​are acquired. After data preprocessing, a graph structure input vector matrix is ​​constructed, and the constructed GCN-Critic model structure is trained iteratively. Depending on the control target values, the final multi-objective control voltage reactive power model and active power command model, which can automatically generate control commands based on real-time grid conditions, are obtained. The voltage reactive power model and active power command model include a GCN strategy network and a Critic evaluation network. The GCN strategy network receives the graph structure input vector matrix and outputs the control action commands for the distribution areas. The Critic evaluation network outputs scores to quantitatively evaluate the multi-objective control performance after executing control action commands under the current grid conditions. During online control, the current distribution network measurement data and control target value are obtained to determine the current control mode; Select the voltage reactive power model / active power command model according to the current control mode. Based on the real-time acquired distribution network measurement data, real-time control target value and preset distribution network adjacency matrix, construct the graph structure input vector matrix at the current moment, input the trained voltage reactive power model / active power command model, and the GCN strategy network calculates based on the learned mapping relationship to output the optimal control command set for controllable nodes, so as to realize real-time optimized control of the transformer area. During model training, the GCN policy network receives a graph-structured input vector matrix and outputs control action commands for the transformer substations, including: The first graph convolutional layer is used based on the distribution network adjacency matrix. and node feature matrix Local feature aggregation is achieved, and one-hop neighborhood aggregation is performed; the second graph convolutional layer is used to perform aggregation based on the distribution network adjacency matrix. and the first layer output features Global feature extraction is achieved, and the receptive field is expanded to a two-hop neighborhood to obtain the second layer of output features. ;include: ； ； , This represents the weight matrices for the first and second graph convolutional layers. for Activation function; For standardized distribution network adjacency matrix; Second layer output features Compressed into a single global feature vector using global average pooling. ; ,in, To represent the total number of nodes in the distribution network; Single global feature vector After nonlinear transformation by a fully connected layer, the Gaussian distribution parameters of the GCN policy network are output based on the transformed vector, including the mean vector and standard deviation vector of the actions. Based on these Gaussian distribution parameters, the policy of the GCN policy network is determined. According to the strategy Unbounded action samples are obtained by sampling the commands and actions issued by the device. The hyperbolic tangent function is used to transform unbounded action samples. Compressing to the [-1,1] interval yields the control command range. ; Wherein, the vector after nonlinear transformation is , This is the weight matrix of the fully connected layer. For bias terms of fully connected layers; Gaussian distribution parameters include , The GCN policy network monitors the power grid status. Mapped to obtain, This is the mean vector of the actions; The standard deviation vector of the action; The bias vector for the mean output layer; The standard deviation is the output layer bias vector; This represents the weight matrix of the mean output layer. The standard deviation represents the output layer weight matrix; Based on Gaussian distribution parameters, determine the policy of the GCN policy network. as follows: ； The sampling of command actions is performed using the following formula: ; Where ⊙ represents element-wise multiplication; It is standard normal random noise.

2. The multi-objective collaborative optimization control method for transformer substations according to claim 1, characterized in that, During model training, the target values ​​for regulation include the target voltage value and the target active power value of the upstream medium-voltage feeder connection node.

3. The multi-objective collaborative optimization control method for transformer substations according to claim 1, characterized in that, During online control, the current control mode is determined based on the real-time control target value. If the real-time control target value is to track the target value of the upper-level feeder voltage, the voltage reactive power model is loaded and enabled. If the real-time control target value is to track the active power target value of the upstream feeder, then the active power command model is loaded and enabled.

4. The multi-objective collaborative optimization control method for transformer substations according to claim 1, characterized in that, During online control, after outputting the optimal control instruction set for controllable nodes and optimizing the control of the transformer area, the following steps are also included: Continuous monitoring acquires operational data after regulation and control and feeds it back to the data center; newly generated operational data is stored and used for periodic incremental training or fine-tuning of the GCN-Critic model, enabling the GCN-Critic model to adapt to changes in power grid structure and evolution of operational characteristics, and achieve self-evolution.

5. The multi-objective collaborative optimization control method for transformer substations according to claim 1, characterized in that, During model training, a graph-structured input vector matrix is ​​constructed, including: Construct the quadruplet input vector data: ， in, The input vector matrix represents the graph structure. For a set of nodes, corresponding to the distribution network One physical node; Let the set of edges represent electrical connections; The node feature matrix integrates multi-dimensional operational status information. This is the adjacency matrix of the distribution network, describing the network topology. Each node Corresponding to a feature vector ,in For feature dimensions; ； in, Injecting active power into the node; Inject reactive power into the node; The node voltage amplitude is the direct indicator of voltage quality. Energy state of the energy storage system; This is a connection identifier for the upstream feeder; This is the target value set by the higher authorities. For the upstream medium-voltage feeder connection node, mark it as ; ， ； in, The target value for voltage regulation at the connection point of the medium-voltage feeder; The target value for active power regulation at the connection point of the medium-voltage feeder; , These represent the target voltage and target power values ​​at the connection node of the upstream medium-voltage feeder, respectively. For non-connected nodes , ; The node feature matrix is ​​constructed based on the node order as follows: ; Distribution network adjacency matrix Based on the physical topology of the distribution network, its elements Defined as: Among them, nodes With nodes The value is 1 for direct electrical connection and 0 for other cases.

6. The multi-objective collaborative optimization control method for transformer substations according to claim 1, characterized in that, The reward function of the voltage reactive power model during model training. From voltage error term Network loss minimization item Root node voltage tracking term The result is obtained by weighting the sum according to different weighting coefficients: ； in, , and These are the weighting coefficients, weighting coefficients , and Adjustments are made based on control priority, with a total of 1. Voltage error term ; Network loss minimization term ; Root node voltage tracking term ; in, Indicates time node The voltage; This represents the set of non-root nodes, that is, the set of all nodes excluding those connected to the upper-level medium-voltage feeder. Represents the voltage-reactive power model Real-time active power loss of the entire network. Indicates time branch road The line current; Indicates the branch resistance; Represents the set of branches; This refers to the root node voltage amplitude connected to the upstream medium-voltage feeder; Set the target voltage value for the root node; To track the intensity coefficient.

7. The multi-objective collaborative optimization control method for transformer substations according to claim 1, characterized in that, During model training, the active instruction model reward function From voltage error term Network loss minimization item Root node active power tracking item Controlling costs and penalties The result is obtained by weighted summation using different weighting coefficients: ； in, > > > The sum is 1; Voltage error term ; Network loss minimization term ; Root node active power tracking term ; Control cost penalty items , ; in, Indicates time node The voltage; and These represent the upper and lower limits of the allowable voltage, respectively. Represents a set of nodes. Indicating active power instruction model Real-time active power loss of the entire network. Indicates time branch road The line current; Indicates the branch resistance; Represents the set of branches. Indicates time The active power of the root node connected to the upper-level medium-voltage feeder; The target value for active power control at the root node; To track the intensity coefficient, Represents a node At any moment The active power regulation amount, , Representing time respectively , Active power; Represents a node Adjustment weighting coefficient, weighting coefficient The nodes are categorized according to their importance, and sorted by priority: critical load nodes, ordinary load nodes, and adjustable resource nodes.

8. A computer device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the multi-objective collaborative optimization control method for transformer substations as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multi-objective collaborative optimization control method for transformer substations as described in any one of claims 1-7.

Images