Virtual power plant voltage-reactive power coordination control method and system based on fusion of graph neural network and deep reinforcement learning

Abstract

The invention discloses a virtual power plant voltage-reactive power coordination control method based on fusion of a graph neural network and deep reinforcement learning, and the method comprises the steps: explicitly introducing a topological structure of a power distribution network into the state representation of deep reinforcement learning in the form of graph data, and extracting and processing the graph structure data through the graph neural network; topological correlation and dynamic interaction among nodes in a power grid are accurately captured; and the extracted graph features and the traditional system state quantity are fused and input to the deep Q network for learning and decision making, so that the optimal coordination control of the distributed resources in the virtual power plant and the power distribution network equipment is output. The virtual power plant voltage-reactive power control method and device aim at solving the problem that in the prior art, perception on a power grid structure is insufficient, and the decision-making precision and robustness of virtual power plant voltage-reactive power control and the adaptability in a large-scale complex power distribution network are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart grid operation control, in particular to a virtual power plant voltage-reactive power coordination control method and system based on the fusion of graph neural network and deep reinforcement learning. BACKGROUND

[0002] With the deepening of energy structure transformation, the penetration rate of distributed renewable energy represented by photovoltaic power generation and wind power generation in the distribution network continues to rise. The inherent intermittency and volatility of such distributed energy pose a serious challenge to the safe and stable operation of the distribution network, with voltage out-of-limit and reactive power imbalance problems being particularly prominent, seriously threatening power quality and grid equipment safety.

[0003] Virtual power plant (VPP) as an advanced resource aggregation and coordinated control technology, through information-physical fusion means, organically aggregates geographically dispersed distributed energy resources (DERs) such as photovoltaic, wind power, energy storage systems and controllable loads, with devices in the distribution network (such as on-load tap changer, capacitor bank), forming a virtual entity of an external dispatchable and internally coordinated power plant.

[0004] Existing deep reinforcement learning (DRL) methods have made some progress in voltage-reactive power control of the power grid, but usually treat grid data as vectors, ignoring the graph structure characteristics of the distribution network. In the virtual power plant scenario, this neglect is more serious because the connection relationship between virtual power plant resources and grid nodes is complex and needs to be explicitly modeled to associate topology. Traditional methods are sensitive to noise and outliers, and it is difficult to ensure that key parameters such as voltage do not exceed the limit, and there is still a gap between theoretical research and practical application. SUMMARY

[0005] The main purpose of the present application is to provide a virtual power plant voltage-reactive power coordination control method based on the fusion of graph neural network and deep reinforcement learning, comprising the following steps:

[0006] Real-time acquisition of the operating state data of the distribution network, and modeling the distribution network as a topology graph, wherein the nodes represent busbars and the edges represent lines, and each node and edge is assigned a node feature vector and an edge feature vector containing real-time measurement data;

[0007] Inputting the topology graph into a pre-trained GNN module and generating a global graph feature vector that integrates topology information;

[0008] The global graph feature vector is spliced with a system numerical state vector not containing topology information to form a fusion state vector;

[0009] The fusion state vector is input into a deep network DQN, and the DQN outputs a Q value for coordinated control of internal distributed resources and power distribution network equipment of the virtual power plant;

[0010] An optimal coordinated control action is selected and executed according to the Q value, and an instant reward is obtained according to a preset multi-objective reward function;

[0011] A four-tuple data composed of a current state, a selected action, an instant reward and a next state is stored in an experience replay buffer, and network parameters of the DQN are updated using data in the experience replay buffer, and the above steps are repeated until the DQN converges, and a stable coordinated control strategy of the virtual power plant is obtained, which is used for coordinated control of internal distributed resources and power distribution network equipment of the virtual power plant.

[0012] In an embodiment, the running state data of the power distribution network is collected in real time, and the power distribution network is modeled as a topology graph, wherein the nodes represent the busbars and the edges represent the lines, and a node feature vector and an edge feature vector containing real-time measurement data are respectively assigned to each node and edge, and the step includes:

[0013] The power grid topology structure is defined, the busbars and lines in the power distribution network are determined, and a topology graph G=(V,E) representing the power grid structure is constructed, wherein V represents a node set in the power grid, E represents an edge set in the power grid, each node vi∈V corresponds to a busbar in the power grid, and each edge eij∈E represents a power line connecting two busbars;

[0014] The feature vector of each node vi contains real-time measurement data related to the busbar, such as voltage amplitude, phase angle, injected active power and reactive power;

[0015] The feature vector of each edge eij describes the line properties and real-time operating parameters between the two busbars, such as line impedance, active power and reactive power.

[0016] In an embodiment, the step of inputting the topology graph into a pre-trained GNN module and generating a global graph feature vector that integrates the correlation information of the power grid topology includes:

[0017] The constructed power grid topology map is input into the pre-trained GNN module. The GNN module iteratively aggregates the feature information of each node and its neighboring nodes through an L-layer message passing mechanism. In each layer, the embedding vector of each node is updated according to the embedding of its previous layer and the embedding of its neighboring nodes, thereby capturing the topological context information from local to global layer by layer. Finally, an L-layer node embedding vector rich in its k-order neighborhood topological association information is generated for each node.

[0018] A global readout function is applied to the set of all L-layer node embedding vectors output by the GNN module. This global readout function is used to aggregate a variable number of node embeddings into a fixed-dimensional vector.

[0019] The aggregation operation of the global readout function generates a global graph feature vector that can characterize the entire power grid topology and its dynamic interactions. This vector is the final output that incorporates the power grid topology information.

[0020] In one embodiment, the step of concatenating the global graph feature vector with the system numerical state vector that does not contain topological information to form a fused state vector includes:

[0021] Obtain the global graph feature vector output by the graph neural network module. This vector is a fixed-dimensional embedding obtained by global pooling operation after multi-layer message passing and aggregation of the distribution network topology graph, and includes the electrical coupling relationship between nodes and the overall operation status.

[0022] Simultaneously acquire a set of system numerical state vectors that do not directly depend on the power grid topology;

[0023] The global graph feature vector and the system numerical state vector are directly concatenated along the feature dimension to form a high-dimensional fused state vector.

[0024] In one embodiment, the system numerical state vector includes, but is not limited to, the current state of all controllable devices, node voltage amplitude, node current injection, node active power injection, node reactive power injection, line active power flow, line reactive power flow, on-load tap changer ratio, capacitor bank switching status, distributed generator output power, energy storage system state of charge, and controllable load status.

[0025] In one embodiment, the step of inputting the fused state vector into a deep QN network, and having the DQN output the Q value for coordinated control of the internal distributed resources and distribution network equipment of the virtual power plant, includes:

[0026] The fused state vector is used as the input to a deep network DQN, which is a feedforward neural network with multiple fully connected layers and network parameters θ.

[0027] The fusion state vector is received by an input layer of the DQN and sequentially undergoes nonlinear feature transformation by at least one hidden layer;

[0028] An output layer of the DQN outputs a corresponding Q value estimate for each executable action in the action space.

[0029] In an embodiment, the step of selecting an optimal coordinated control action according to the Q value and performing, while obtaining an immediate reward according to a preset multi-objective reward function, comprises:

[0030] Based on the Q value corresponding to each action output by the DQN, an action is selected using an ε-greedy strategy, wherein an action with the maximum Q value is selected as the current optimal coordinated control action with a probability of 1-ε, and an action is randomly selected to maintain exploration ability with a probability of ε;

[0031] The selected current optimal coordinated control action is parsed into specific device control instructions and is synchronously issued to distributed resources inside the virtual power plant and devices in the power distribution network, to complete a coordinated control operation;

[0032] After the optimal coordinated control action is executed, running state data at the next time is collected from the distributed resources inside the virtual power plant and the devices in the power distribution network;

[0033] Based on the running state data at the next time, an immediate reward value is calculated according to a preset multi-objective reward function.

[0034] In an embodiment, the multi-objective reward function r_t is designed as: r_t = -J_t, wherein J_t is a comprehensive cost function at time t, and the comprehensive cost function J_t includes a weighted sum of one or more of the following: a voltage deviation penalty term, a power factor correction term, a network loss penalty term, a device action frequency penalty term, and a virtual power plant operation cost term.

[0035] In an embodiment, a four-tuple data composed of a current state, a selected action, an immediate reward, and a next state is stored in an experience replay buffer, and data in the experience replay buffer is used to update network parameters of the DQN, and the above steps are repeated until the DQN converges, to obtain a stable coordinated control strategy of the virtual power plant, and the step of using the coordinated control strategy to perform coordinated control on the internal distributed resources of the virtual power plant and the devices in the power distribution network comprises:

[0036] After each coordination control action cycle, the current fusion state vector st, the selected optimal coordination control action at, the calculated immediate reward rt and the next fusion state vector st+1 observed after the action are packaged into an experience quadruple (st, at, rt, st+1) and stored in an experience replay buffer with a preset capacity;

[0037] When the number of samples in the experience replay buffer reaches a preset minimum training threshold, the update process of the DQN network parameters is started;

[0038] When the loss function of the DQN network converges to a stable level, and the performance index in the verification scene reaches a preset threshold and remains stable, it is determined that the training is completed, and the strategy finally obtained by the DQN is a stable virtual power plant voltage-reactive power coordination control strategy.

[0039] A virtual power plant voltage-reactive power coordination control system based on the fusion of graph neural networks and deep reinforcement learning, the system comprising:

[0040] A data acquisition and graph construction unit configured to acquire power distribution network data in real time and construct the topological graph;

[0041] A GNN feature extraction unit embedded with a pre-trained and weight-frozen GNN module, configured to receive the topological graph and output the global graph feature vector;

[0042] A state fusion unit configured to concatenate the global graph feature vector and the system numerical state vector into the fusion state vector;

[0043] A DQN decision unit configured to receive the fusion state vector, output the Q value of each control action, and select the optimal action;

[0044] A control execution unit configured to issue the optimal action to the corresponding distributed resources and power distribution network equipment;

[0045] An experience replay and learning unit configured to store interaction experience, calculate rewards and sample data, and update the parameters of the DQN decision unit.

[0046] Therefore, the present application has the following advantages:

[0047] The application provides a virtual power plant voltage-reactive power coordinated control method based on the fusion of graph neural network and deep reinforcement learning, including the following steps: collecting the operation state data of the power distribution network in real time, modeling the power distribution network as a topology graph, wherein the nodes represent busbars, the edges represent lines, and the node feature vectors and edge feature vectors containing real-time measurement data are respectively assigned to each node and edge; inputting the topology graph into a pre-trained GNN module and generating a global graph feature vector fused with topology information; splicing the global graph feature vector and a system numerical state vector not containing topology information to form a fusion state vector; inputting the fusion state vector into a deep network DQN, outputting the Q value of the coordinated control of the internal distributed resources of the virtual power plant and the power distribution network equipment by the DQN; selecting the optimal coordinated control action according to the Q value and executing it, while obtaining the immediate reward according to the preset multi-objective reward function; storing the four-tuple data composed of the current state, the selected action, the immediate reward and the next state in the experience replay buffer, and updating the network parameters of the DQN by using the data in the experience replay buffer, repeating the above steps until the DQN converges, and obtaining a stable coordinated control strategy of the virtual power plant, which is used for the coordinated control of the internal distributed resources of the virtual power plant and the power distribution network equipment. The application aims to solve the problem of insufficient perception of the power grid structure in the prior art, combines the understanding ability of the graph neural network to the non-Euclidean space topology relationship with the sequence decision advantage of the deep reinforcement learning, and utilizes the power grid topology structure information to deeply fuse the physical mechanism and data-driven intelligence, realizes the real-time, accurate, robust and safe collaborative optimization control of the internal resources of the virtual power plant and the power distribution network equipment, and effectively copes with the voltage stability and reactive power optimization challenges of the power distribution network under the background of high proportion of distributed energy access. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can also be obtained by those skilled in the art without any creative effort.

[0049] Figure 1 It is a system flow chart of the virtual power plant voltage-reactive power coordinated control method based on the fusion of graph neural network and deep reinforcement learning.

[0050] Figure 2 It is an experimental effect comparison schematic diagram of the virtual power plant voltage-reactive power coordinated control method based on the fusion of graph neural network and deep reinforcement learning. DETAILED DESCRIPTION

[0051] In order to make the purposes, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely in conjunction with the accompanying drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0052] It should be understood that the specific embodiments described herein are used only to explain the present application and not to limit the present application.

[0053] In view of the defects in the prior art, the present application provides a virtual power plant voltage-var coordination control method based on the fusion of graph neural network and deep reinforcement learning. The topological structure of the power distribution network is explicitly introduced into the state representation of deep reinforcement learning in the form of graph data. The graph neural network is used to extract and process the graph structure data to accurately capture the topological correlation and dynamic interaction between nodes in the power grid. Then the extracted graph features and traditional system state quantities are fused and input into the deep Q network for learning and decision-making, so as to output the optimal real-time control instructions for the distributed resources and power distribution network equipment in the virtual power plant. The core value of the virtual power plant lies in its ability to act as an intelligent agent between the power distribution network and the distributed resources. Through internal coordination and control strategies, it participates in the voltage-var control (VVC) of the power grid to maintain the node voltage within the safety limit, reduce network loss, improve power quality, and optimize its own operation economy. The graph neural network (GNN) is a neural network model specially designed for processing graph structure data. In GNN, each node is considered as an independent learning unit, and its state is not only affected by its own features, but also by the states of its neighboring nodes. Through multiple rounds of information transmission process, each node can gradually aggregate information from its neighbors and update its own state accordingly. This information propagation mechanism enables GNN to effectively extract the structural information in the graph and encode it into the node embedding representation. These embeddings can be used for various downstream tasks such as node classification, link prediction, subgraph identification, etc.

[0054] Specific to the virtual power plant voltage-reactive power coordination control method, the application of GNN is particularly important, because the distribution network is essentially a complex graph structure, in which the nodes represent the bus and the edges represent the line. The traditional numerical-based state representation method ignores the topological structure characteristics of the power grid, while GNN can more accurately capture the topological correlation and dynamic interaction between nodes in the power grid by modeling the power grid as a graph and using node feature vectors and edge feature vectors to represent the real-time measurement data of each bus and line. In this way, GNN can provide more rich and accurate state representation in the deep reinforcement learning framework, thereby improving the decision accuracy and robustness of the control strategy.

[0055] The embodiment of the present application provides a virtual power plant voltage-reactive power coordination control method based on the fusion of graph neural network and deep reinforcement learning, which comprises steps S10-S60, referring to Figure 1 , Figure 1 The embodiment of the present application provides a virtual power plant voltage-reactive power coordination control method based on the fusion of graph neural network and deep reinforcement learning, which comprises steps S10-S60, referring to

[0056] Step S10: Real-time acquisition of the operation state data of the distribution network, and modeling the distribution network as a topological graph, wherein the nodes represent the bus and the edges represent the line, and the node feature vectors and edge feature vectors containing real-time measurement data are respectively assigned to each node and edge;

[0057] Step S20: inputting the topological graph into a pre-trained GNN module and generating a global graph feature vector fused with topological information;

[0058] Step S30: concatenating the global graph feature vector with a system numerical state vector not containing topological information to form a fusion state vector;

[0059] Step S40: inputting the fusion state vector into the deep network DQN, and outputting the Q value of the coordination control of the internal distributed resources of the virtual power plant and the distribution network equipment from the DQN;

[0060] Step S50: selecting the optimal coordination control action according to the Q value and executing it, and simultaneously obtaining the immediate reward according to the preset multi-objective reward function;

[0061] Step S60: storing the four-tuple data composed of the current state, the selected action, the immediate reward and the next state in the experience replay buffer, and updating the network parameters of the DQN using the data in the experience replay buffer, repeating the above steps until the DQN converges, obtaining a stable virtual power plant coordination control strategy for coordinating the internal distributed resources of the virtual power plant and the distribution network equipment.

[0062] Specifically, in the embodiment, the virtual power plant voltage-reactive power coordination control method based on the fusion of the graph neural network and the deep reinforcement learning has a core of constructing a GNN-QL intelligent control framework of a "perception-decision-learning" closed loop, and is specifically implemented through the following six steps.

[0063] Step S10: topology graph construction and state perception

[0064] First, real-time operation data are collected from a distribution network SCADA, a PMU or a simulation platform, including bus voltages, line powers, device states and the like. Then, the physical structure of the distribution network is explicitly modeled as a topology graph. In the graph, nodes correspond to buses in the power grid, and node feature vectors are assigned to real-time measurement data of the buses, such as voltage amplitudes, active / reactive power injection powers; edges correspond to electrical connections (such as transmission lines) between buses, and edge feature vectors include inherent parameters of the lines, such as resistances and reactances. This graph structure naturally retains the non-Euclidean space characteristics of the power grid, laying a foundation for subsequent topology perception.

[0065] Step S20: topology feature extraction based on GNN

[0066] The topology graph constructed in step S10 is input into a pre-trained GNN network. The GNN network (preferably a graph attention network) learns the embedding representation of each node in the global topology by aggregating the multi-hop neighbor information of each node through its message passing mechanism. Then, through global average pooling and the like, all node embeddings are compressed into a fixed-dimensional global graph feature vector. The vector is a precise mathematical representation of the correlation and dynamic interaction of the power grid topology, and is rich in structural information that cannot be expressed by traditional vectors.

[0067] Step S30: fusion state vector construction

[0068] In order to take into account the topology information and the system-level macro state, the global graph feature vector obtained in step S20 is spliced with a group of system numerical state vectors that do not contain topology information. The latter includes system frequency, VPP aggregated total power prediction value, key tie-line power flow and other global indicators. Through this splicing operation, a high-dimensional fusion state vector is formed, which perceives both local electrical connection relationships and global operation situations, and provides comprehensive information input for intelligent decision-making.

[0069] Step S40: DQN-driven coordination control decision

[0070] The fusion state vector is input to a deep Q network (DQN). The DQN is a multi-layer fully connected neural network, and the dimension of the output layer corresponds to a predefined joint action space. The action space is composed of the actions of the internal resources of the VPP (such as the reactive power level of the photovoltaic inverter, the charging and discharging power of the energy storage) and the power distribution network equipment. The DQN outputs a Q value for each possible joint action through forward calculation, and the value represents the long-term cumulative reward expected to be obtained after the action is performed.

[0071] Step S50: Action execution and immediate reward calculation

[0072] According to the Q value output by the DQN, an optimal coordinated control action is selected by using an ε-greedy strategy or the like, and the action is parsed into specific device instructions and executed. After the action is executed, the system calculates an immediate reward according to a preset multi-objective reward function. The function comprehensively considers multiple objectives such as voltage deviation, network loss, device action frequency, and VPP coordination cost, and guides the agent to learn a comprehensive optimal strategy that takes into account safety, economy, and device life through negative weighted summation.

[0073] Step S60: Experience replay and policy learning

[0074] The four-tuple data (current state, selected action, immediate reward, and next state) of this interaction is stored in an experience replay buffer. When the buffer samples are sufficient, a batch of data is randomly sampled from the buffer, a stable learning target is calculated using a target network, and the network parameters of the DQN are updated by minimizing the Bellman error. By periodically synchronizing the parameters of the main network and the target network, the training process is stable. Repeat the closed-loop process of S10 to S60, and the DQN is continuously optimized in the continuous interaction with the environment until it converges, and finally a stable coordinated control strategy is obtained for online real-time control of VPP resources and power grid equipment, achieving high-precision and strong-robust voltage-reactive power collaborative optimization.

[0075] Further, in the embodiment, the operation state data of the power distribution network is collected in real time, and the power distribution network is modeled as a topology graph, wherein the nodes represent busbars and the edges represent lines, and each node and edge is assigned a node feature vector and an edge feature vector containing real-time measurement data. The step includes:

[0076] The topology structure of the power grid is defined, the busbars and lines in the power distribution network are determined, and a topology graph G=(V,E) representing the structure of the power grid is constructed, where V represents the node set of the power grid, E represents the edge set of the power grid, each node vi∈V corresponds to a busbar in the power grid, and each edge eij∈E represents a power line connecting two busbars.

[0077] The feature vector of each node vi contains real-time measurement data related to that bus, such as voltage amplitude, phase angle, injected active power and reactive power;

[0078] The eigenvector of each edge eij describes the line attributes and real-time operating parameters connecting the two buses, such as line impedance, active power, and reactive power.

[0079] Specifically, in this embodiment, to achieve voltage-reactive power coordinated control of the virtual power plant, it is first necessary to define and model the power grid topology. Clearly defining the buses and lines in the distribution network is the foundation for constructing the power grid topology diagram. A distribution network typically consists of multiple buses and power lines connecting these buses. Buses, as key nodes in power distribution, play a role in power aggregation and distribution; while power lines are responsible for transmitting power between different buses. Based on this, a topology diagram G=(V,E) representing the power grid structure is constructed, where V represents the set of nodes in the power grid, and E represents the set of edges in the power grid. Each node vi∈V corresponds to a bus in the power grid, and each edge eij∈E represents a power line connecting two buses. In this way, the physical power grid is abstracted into an easily tractable mathematical model, specifically including:

[0080] Node electrical quantities: Node voltage amplitude Current injection Active power injection and reactive power injection ,in, and This may include the injected power of virtual power plant resources.

[0081] Line electrical quantities: connection nodes and Active power flow on the line and reactive power flow .

[0082] On-load tap changer (OLTC): adjusts the turns ratio To manage voltage levels, the physical relationships are as follows:

[0083]

[0084] in, It is a node voltage amplitude, It is the ratio of OLTC. and These are the resistance and reactance of the circuit, respectively. and is the active and reactive power flow on the line, is the squared magnitude of the line current;

[0085] Capacitor bank: at node , the capacitor is switched through binary states to provide reactive support, whose output power is proportional to the square of the node voltage, modelled as:

[0086]

[0087] where, is the reactive output of the capacitor at node , is the switching state (0 for off, 1 for on), is the maximum reactive capacity of the capacitor, is the voltage at node ;

[0088] Virtual power plant resources: for distributed resources in the virtual power plant, the control variables include active power output and reactive power output , which can be part of the control action.

[0089] Further, in the embodiment, the step of inputting the topology graph into a pre-trained GNN module and generating a global graph feature vector that integrates grid topology correlation information comprises:

[0090] The constructed grid topology graph is input into a pre-trained GNN module. The GNN module iteratively aggregates the feature information of each node and its neighbor nodes through an L-layer message passing mechanism. In each layer, the embedding vector of each node is updated according to its own previous layer embedding and the embedding of its adjacent nodes, thereby capturing the topology context information from local to global layer by layer, and finally generating an L-layer node embedding vector for each node that is rich in k-order neighborhood topology correlation information.

[0091] A global readout function is applied to the set of all L-layer node embedding vectors output by the GNN module. The global readout function is used to aggregate a variable number of node embeddings into a fixed-dimensional vector.

[0092] Through the aggregation operation of the global readout function, a global graph feature vector is generated that can represent the entire grid topology structure and its dynamic interaction. This vector is the final output that integrates grid topology information.

[0093] Specifically, in this embodiment, in order to effectively integrate the topology and dynamic characteristics of the power grid into the voltage-reactive power coordination control strategy, a graph neural network is used to process the constructed power grid topology graph. This process not only captures the local relationships between nodes in the power grid, but also extracts global topological correlation information, providing strong support for subsequent deep reinforcement learning decisions.

[0094] First, the constructed power grid topology graph is fed into a pre-trained GNN module as input. The core of this module is its message passing mechanism, which aggregates the feature information of each node and its neighbor nodes layer by layer through L iterations. Specifically, in each layer, the embedding vector of each node is updated based on its own embedding in the previous layer and the embedding of neighboring nodes. This mechanism enables the representation of a node to depend not only on its own attributes but also on information from its neighborhood. As the number of layers increases, the embedding vector of each node gradually contains topological correlation information from first-order neighbors to k-order neighbors, thus realizing the transition from a local perspective to a global perspective.

[0095] In each layer, GNN performs a message passing operation, including three main steps: message generation, message aggregation, and node update. For each node vi, first, a message is generated based on its current embedding vector and the features of the edges connected to it; then, these messages are aggregated into a new vector representing information from the neighborhood; finally, this new vector is combined with the original embedding of vi to produce an updated embedding vector. In this way, GNN can integrate more extensive contextual information at each step, ultimately generating an L-layer node embedding vector for each node that is rich in k-order neighborhood topological correlation information. This enables each node to perceive part or all of the topology of the entire power grid, greatly enhancing the expressiveness and adaptability of the model.

[0096] Next, a global readout function is applied to the set of all L-layer node embedding vectors output by the GNN module. The function's role is to aggregate a variable number of node embeddings into a fixed-dimensional vector. This is because the size and complexity of the power grid may vary, resulting in different numbers of nodes, while the subsequent algorithm for optimizing the control strategy requires a fixed-size input. Global readout functions usually use pooling operations such as global average pooling or global max pooling, and sometimes combine attention mechanisms to weight the importance of each node. This approach not only effectively handles power grid data of different sizes, but also highlights the role of key nodes, improving the robustness and generalization ability of the model.

[0097] Through the aggregation operation of the global readout function described above, a global graph feature vector capable of representing the entire power grid topology and its dynamic interaction is generated. This vector integrates the topology information of the power grid, containing rich information about the connection mode of the power grid, line parameters, and their real-time operating state. It not only reflects the static structure of the power grid, but also embodies the dynamic operating characteristics.

[0098] Further, in the present embodiment, the step of splicing the global graph feature vector and the system numerical state vector not containing topology information to form a fusion state vector includes:

[0099] Obtaining the global graph feature vector output by the graph neural network module, which is a fixed-dimensional embedding obtained through global pooling operation after multi-layer message passing and aggregation on the distribution network topology graph, and contains the electrical coupling relationship between nodes and the overall operating situation;

[0100] Synchronously collecting a group of system numerical state vectors not directly dependent on the topology structure of the power grid;

[0101] Directly splicing the global graph feature vector and the system numerical state vector in the feature dimension to form a high-dimensional fusion state vector.

[0102] Specifically, in the present embodiment, in order to effectively combine the topology structure information of the power grid with the system numerical state, a method of splicing the global graph feature vector and the system numerical state vector not containing topology information is adopted to support the formulation of the virtual power plant voltage-reactive power coordination control strategy. This process not only strengthens the understanding of the operating state of the power grid, but also provides more comprehensive and accurate input data for the deep reinforcement learning algorithm.

[0103] First, the global graph feature vector output by the graph neural network needs to be obtained. This vector is a fixed-dimensional embedding obtained through global pooling operation after multi-layer message passing and aggregation on the distribution network topology graph. This process fully utilizes the advantages of GNN in handling non-Euclidean space data, and can capture the electrical coupling relationship between nodes and the overall operating situation. Specifically, GNN updates the embedding vector of each node layer by layer through L-layer message passing mechanism, so that these vectors not only contain the attribute information of the node itself, but also integrate the information of other nodes in its neighborhood. Subsequently, a global readout function (such as global average pooling or global maximum pooling) is used to aggregate the final embedding vectors of all nodes to generate a fixed-length global graph feature vector. This vector is essentially a high-level abstraction of the entire power grid topology and its dynamic characteristics, providing key topological correlation information for subsequent decision-making.

[0104] Meanwhile, a set of system numerical state vectors are collected synchronously, which are not directly dependent on the grid topology. These state variables typically include, but are not limited to, real-time operating parameters such as actual measured voltages, active power, and reactive power of each node. It is worth noting that these numerical states do not directly reflect the physical connection of the grid, but they are crucial for understanding the current operating conditions of the grid. For example, some nodes may be experiencing high load conditions, leading to local voltage drops; or due to weather changes, the power generation of renewable energy sources has fluctuated significantly. Therefore, collecting and analyzing these numerical states helps to more accurately assess the overall health of the grid and timely identify potential problems.

[0105] Next, the global graph feature vector is directly concatenated with the system numerical state vector in the feature dimension to form a high-dimensional fusion state vector. This operation aims to integrate information from two different sources, preserving the important characteristics of the grid topology while considering the actual operating state of the system. In this way, a more comprehensive and detailed state representation can be created, providing strong support for subsequent intelligent control strategies. In implementation, the two vectors can be simply arranged in dimension order to combine into a new vector, or specific weight allocation schemes can be designed according to the needs of specific application scenarios to adjust the relative importance between the two.

[0106] Further, in this embodiment, the system numerical state vector includes but is not limited to the current state of all controllable devices, node voltage amplitude, node current injection, node active power injection, node reactive power injection, line active power flow, line reactive power flow, on-load tap changer ratio, capacitor bank switching state, distributed generator output power, energy storage system state of charge, and controllable load state.

[0107] Specifically, in this embodiment, the system numerical state vector is a key component of building a fusion state representation, and its design aims to comprehensively depict the operating conditions of the distribution grid at the current time, providing high-dimensional and accurate numerical input for subsequent intelligent decision-making. This vector does not directly encode the topological connection of the grid, but focuses on observable physical quantities that reflect the dynamic behavior and device state of the system. Specifically, its content includes but is not limited to the following elements:

[0108] First, the current state of all controllable devices, i.e., the real-time operation mode or action position of adjustable resources inside the virtual power plant and on the distribution network side, is covered, which is the basis for closed-loop control. Second, the voltage amplitude of each node is included as a core indicator for measuring power quality and voltage stability; node current injection, node active power injection, and node reactive power injection are also collected to fully describe the power balance state of each bus. On the line level, the system synchronously acquires line active power flow and line reactive power flow for evaluating line load level and network loss.

[0109] In addition, for traditional voltage regulation devices, the state vector explicitly includes the on-load tap changer, reflecting its current voltage regulation position; and the capacitor bank switching state (such as 0 for removal and 1 for input), representing the operation of the reactive power compensation device. For the distributed resources aggregated by the virtual power plant, the state vector is further refined into distributed generator output power (such as real-time active output of photovoltaic and wind power) and state of charge (SOC) of energy storage system, which are used to determine the available adjustment margin of energy storage and the state of controllable load.

[0110] All the above numerical quantities together constitute a high-dimensional, structured system numerical state vector. Although this vector does not explicitly contain topology information, each dimension is closely related to the actual operation of the power grid. By concatenating it with the global graph feature vector extracted by the graph neural network, the deep fusion of topology structure perception and operation state quantification can be achieved, providing the deep Q network with a state representation that is both physically reasonable and data complete

[0111] Further, in the embodiment, the step of inputting the fusion state vector into the deep network DQN and outputting the Q value of the coordinated control of the internal distributed resources of the virtual power plant and the distribution network devices by the DQN includes:

[0112] The fusion state vector is input into the deep network DQN, and the DQN is a feedforward neural network with multiple fully connected layers, and its network parameters are θ;

[0113] The fusion state vector is received by the input layer of the DQN and is sequentially transformed by at least one hidden layer for nonlinear feature transformation;

[0114] At the output layer of the DQN, a corresponding Q value estimate is output for each executable action in the action space.

[0115] Specifically, in the present embodiment, the process of inputting the fusion state vector into a deep Q-network (DQN) and outputting a coordinated control Q-value is the core step for the GNN-QL intelligent control framework to realize the decision-making function. This step aims to map the high-dimensional state representation that fuses the power grid topology correlation and system operating state into a quantitative evaluation of the long-term benefits of various control actions, thereby providing a basis for optimal action selection.

[0116] Specifically, first, the fusion state vector generated in the preceding step is taken as the only input of the DQN. The DQN is designed as a feedforward neural network with multiple layers, and the overall network parameters are denoted as θ. The input layer dimension of the network is strictly aligned with the dimension of the fusion state vector, ensuring that all information is losslessly entered into the model. Subsequently, the input data passes through at least one hidden layer (usually 2-3 layers) in turn, each layer consisting of multiple neurons and using a nonlinear activation function (such as ReLU) for feature transformation. The role of these hidden layers is to perform layer-by-layer abstraction and high-order feature extraction on the original fusion state, to mine the complex nonlinear coupling relationships between state variables, such as the dynamic correlation between voltage deviation and line loss, the comprehensive influence of device actions on multi-node voltage, etc., thereby constructing a deep understanding of the current power grid operating situation.

[0117] Finally, the information is transmitted to the output layer of the DQN. The number of neurons in this layer is exactly the same as the size of the predefined joint action space. The joint action space is composed of all possible combinations of controllable resources within the virtual power plant (such as the reactive power output level of distributed photovoltaic inverters, the charge / discharge power level of energy storage systems, the regulation level of controllable loads, etc.) and traditional devices in the distribution network (such as the up / down command of the on-load tap changer OLTC, the on / off command of the shunt capacitor bank), which are usually discretized to adapt to the output structure of the DQN. For each executable joint action a in the action space, the output layer of the DQN outputs a corresponding Q-value estimate. This Q-value is not an immediate reward, but the expected discounted sum of future cumulative rewards that the agent can obtain after performing action a in the current fusion state s, and its essence is a quantitative prediction of the long-term comprehensive benefits (including voltage safety, network loss economy, device life, and resource coordination cost, etc.) of the action.

[0118] Through this process, the DQN successfully converts the complex power grid state perception results into value evaluation for specific control actions, laying a solid foundation for subsequent selection of the optimal coordinated control strategy based on Q-values. This design not only ensures the real-time nature of decision-making, but also enables the control strategy to adaptively respond to power grid dynamics through an end-to-end learning mechanism, significantly outperforming traditional methods that rely on artificial rules or simplified models.

[0119] Further, in the embodiment, the step of selecting an optimal coordinated control action according to the Q value and executing, while obtaining an instant reward according to a preset multi-objective reward function, comprises:

[0120] Based on the Q value corresponding to each action output by the DQN, an action is selected using an ε-greedy strategy, in which an action with the maximum Q value is selected as the current optimal coordinated control action with a probability of 1-ε, and an action is randomly selected to maintain the exploration ability with a probability of ε;

[0121] The selected current optimal coordinated control action is parsed into specific device control instructions and is synchronously issued to the distributed resources inside the virtual power plant and the devices in the power distribution network, to complete a coordinated control operation;

[0122] After the optimal coordinated control action is executed, the running state data at the next time is collected from the distributed resources inside the virtual power plant and the devices in the power distribution network;

[0123] Based on the running state data at the next time, an instant reward value is calculated according to the preset multi-objective reward function.

[0124] Specifically, in the embodiment, the step of selecting an optimal coordinated control action according to the Q value and executing, while obtaining an instant reward according to a preset multi-objective reward function, constitutes a key link of the "decision-making-execution-feedback" closed loop in the control method, and the purpose is to balance exploration and utilization, and guide the strategy to continuously optimize through quantitative feedback.

[0125] First, after the DQN completes processing of the current fusion state vector and outputs the Q values corresponding to all possible joint actions, an action is selected using an ε-greedy strategy. This strategy is a classic reinforcement learning exploration mechanism: an action with the maximum Q value, i.e., the optimal coordinated control action under the current strategy, is selected with a high probability of 1-ε (usually ε is a small positive number, such as 0.1, and gradually decays with the training process), which reflects the utilization of existing knowledge; at the same time, an arbitrary action in the action space is randomly selected with a low probability of ε to maintain the exploration ability of unknown state-action pairs, avoid the agent from falling into a local optimum too early, and ensure that the agent has the potential to continuously learn and adapt to new working conditions in a complex and dynamic power grid environment.

[0126] After the selected action, immediately enter the control instruction analysis and issue phase. The selected joint action is an abstract discrete code that needs to be analyzed into executable instructions for specific physical devices in real time. For example, if the action code corresponds to "raise the OLTC by one step, turn on a group of capacitors, and discharge the energy storage system at 0.5 p.u. power", the corresponding control signals will be generated simultaneously, and through the communication interface (such as IEC 61850 or DNP3), they will be issued to the distribution network automation system (control OLTC and capacitors) and the virtual power plant energy management system to schedule energy storage and distributed resources. This process realizes the coordinated and synchronous control of internal resources of the VPP and external grid devices.

[0127] After the control instructions are executed and the grid state tends to be stable (usually at the end of a control period), the next time's operating state data is collected from SCADA, PMU or VPP local monitoring system, including key measurement values such as node voltage, line flow, device actual action state, energy storage SOC, etc. These data are not only used to build the state input of the next time step, but also the basis for calculating the immediate reward.

[0128] Finally, according to the preset multi-objective reward function, based on the above collected new state data, the immediate reward value brought by this control action is calculated. The function is carefully designed as a negative weighted sum, which comprehensively reflects voltage deviation, network loss, device action cost and VPP coordination cost, etc. For example, if all node voltages return to the safe interval and the network loss is significantly reduced after control, the reward value is a large negative number (since the goal is to minimize cost, the reward is negative); on the contrary, if there is voltage out-of-limit or frequent operation of devices, the reward value will deteriorate sharply (i.e. more negative). This immediate reward, as a direct feedback of the environment to the agent's behavior, is used for subsequent experience replay and network parameter update, driving the DQN to continuously learn better control strategies, and ultimately achieving the comprehensive optimization goal of safe, economic and long-life operation.

[0129] Further, in the embodiment, the multi-objective reward function r_t is designed as: r_t = -J_t, where J_t is a comprehensive cost function at time t, and the comprehensive cost function J_t includes one or more of the following weighted sums: voltage deviation penalty term, power factor correction term, network loss penalty term, device action frequency penalty term, and virtual power plant operation cost term.

[0130] Specifically, in the embodiment, the multi-objective reward function rt is carefully designed as the negative value of the comprehensive cost function Jt, that is:

[0131] rt=−Jt

[0132] The design follows the consistency principle of maximizing cumulative rewards and minimizing operating cost objectives in reinforcement learning. By converting the control objective into a cost minimization problem and taking its negative value as the reward, DQN naturally tends to learn a coordinated control strategy that can reduce the overall operating cost of the system during the training process.

[0133] where the comprehensive cost function Jt is a scalar index that evaluates the system's operating performance at time t, and its composition fully embodies the essence of multi-objective optimization of virtual power plant voltage-reactive power coordinated control. Specifically, Jt includes one or more of the following weighted sums:

[0134] Voltage deviation penalty term: used to measure the extent to which all node voltages deviate from the rated value (usually 1.0 p.u.), the common form is the sum of squares of voltage deviation of each node, and the weight of this penalty term is large, which is the core constraint to ensure power quality and system safety, and can effectively suppress the risk of voltage out-of-limit.

[0135] Power factor correction term: reflects the overall reactive power balance level of the system, which can be achieved by penalizing the power factor deviation from the ideal value (such as 0.95 or above) of the main feeder or key nodes, or directly constraining the total amount of reactive power flow, aiming to reduce the ineffective transmission of reactive power in the network and improve system efficiency.

[0136] Network loss penalty term: directly counts the total active power loss at time t, which is proportional to the square of the line current, and is a key indicator of measuring operating economy. By minimizing this term, the operating energy consumption of the distribution network can be effectively reduced.

[0137] Device action frequency penalty term: for devices such as on-load tap changers and capacitor banks, count whether they have acted at time t (such as 1 if they have acted, otherwise 0), and multiply by a high weight coefficient. This penalty term aims to suppress the chattering behavior of the control strategy, prolong the service life of the device, and meet the actual engineering needs.

[0138] Virtual power plant operating cost term: quantifies the coordination cost of calling internal resources of VPP, including but not limited to: depreciation cost of energy storage systems due to charging and discharging cycles, opportunity cost of active power generation that may be lost by distributed power sources (such as photovoltaic) due to providing reactive power support, and compensation fees paid by controllable loads for participating in adjustment, etc. This term ensures that the internal economy of VPP is also considered in the global optimization while providing auxiliary services.

[0139] The comprehensive cost function can be formally expressed as:

[0140]

[0141] Wherein omega1, omega2, …, omega5 >= 0 are adjustable weight coefficients, used to flexibly balance the priority of each sub-target according to the actual operation scene. Through the multi-objective reward function, the application realizes the unified optimization of voltage safety, operation economy, equipment life and VPP internal coordination, so that the agent can adaptively generate a high-quality control strategy considering multiple performance indicators in the autonomous learning process.

[0142] Further, in the embodiment, the four-tuple data composed of the current state, the selected action, the immediate reward and the next state is stored in the experience replay buffer, and the network parameters of the DQN are updated using the data in the experience replay buffer, and the above steps are repeated until the DQN converges, obtaining a stable virtual power plant coordination control strategy, and the step of coordinating and controlling the internal distributed resources and power distribution network equipment of the virtual power plant comprises:

[0143] After each coordination control action cycle ends, the current fusion state vector st, the selected optimal coordination control action at, the calculated immediate reward rt and the observed next fusion state st+1 after executing the action are packaged into an experience four-tuple (st, at, rt, st+1), and stored in the experience replay buffer with a preset capacity.

[0144] When the number of samples in the experience replay buffer reaches the preset minimum training threshold, the update process of the DQN network parameters is started.

[0145] When the loss function of the DQN network converges to a stable level, and the performance index in the verification scene reaches the preset threshold and remains stable, it is determined that the training is completed, and the strategy generated by the DQN finally obtains a stable virtual power plant voltage-reactive power coordination control strategy.

[0146] Specifically, in the embodiment, the step of "storing the experience four-tuple in the experience replay buffer and updating the DQN network parameters until the strategy converges" constitutes the core learning mechanism for realizing continuous optimization and final deployment of the strategy in the GNN-QL framework. Through the introduction of key technologies such as experience replay and target network, this process effectively solves the problems of unstable training and low sample efficiency of reinforcement learning in high-dimensional continuous state space, ensuring the robustness and practicality of the learned strategy.

[0147] Specifically, after each coordination control action cycle, the system encapsulates the key information of this interaction as an experience quadruple (st, at, rt, st+1). Where st is the fusion state vector (combining the topological graph features and system numerical state) at the current time, at is the optimal coordination control action selected based on the DQN output Q value, rt is the immediate reward calculated according to the multi-objective reward function, and st+1 is the next time fusion state vector observed from the power grid environment after the action is executed. This quadruple completely records the closed-loop interaction process of "state-action-feedback-new state", and is stored in an experience replay buffer with a fixed capacity (such as 10000 or 100000). The buffer uses a first-in, first-out strategy to manage, ensuring that the model always learns the latest running experience while preserving historical diversity.

[0148] When the number of samples in the buffer accumulates to the preset minimum training threshold (such as 1000), the update process of the DQN network parameters is started. At this time, a batch (mini-batch, such as 64 or 128) of experience quadruples is randomly sampled from the buffer. Random sampling breaks the strong correlation of data over time, significantly improving the stability and generalization ability of training. For each sampled experience, a target network with the same structure but lagging parameter updates is used. Then, the mean square error between the DQN network's predicted Q value and the target Q value is used to construct the loss function, and the network parameters θ are updated through gradient descent by optimizers such as Adam. The target network parameters θ are periodically updated to provide a stable learning target.

[0149] The closed-loop process described above is repeatedly executed in multiple training rounds. The system continuously monitors the loss function value of the DQN and key performance indicators (such as the average voltage mean square error MSE, voltage out-of-limit rate, network loss, etc.) in an independent validation scenario. When the loss function converges to a lower and stable level, and the validation indicators reach the preset threshold (such as MSE < 0.6, out-of-limit rate < 5%) for consecutive rounds and remain stable, it is determined that the training has converged. At this time, the mapping relationship encoded by the main DQN network is the final obtained stable virtual power plant voltage-reactive power coordination control strategy. This strategy can be deployed in an online control system, which receives real-time grid state inputs and outputs optimal control instructions, achieving efficient and adaptive co-regulation of distributed resources (such as photovoltaic, energy storage, and controllable loads) within the VPP and distribution network equipment (such as OLTC, capacitor banks), thereby comprehensively improving the operational economy and reliability of the distribution network under the premise of ensuring voltage safety. The experiment uses the IEEE13-node standard test system, which includes 13 bus nodes, 9 load points, 1 on-load voltage regulating transformer, and 2 groups of switchable capacitors. The virtual power plant aggregates distributed photovoltaic, wind power, and energy storage systems, and their output power is simulated based on actual weather data. Based on the OpenDSS power system simulation software, a dynamic simulation environment is constructed, and real-time interaction with the reinforcement learning environment is achieved through the Python-OpenDSS interface. The following data is extracted from OpenDSS in real time: (voltage amplitude), (current injection), , (node injection power), , (line power flow), OLTC tap position, capacitor switching state, DG output power, energy storage SOC, controllable load state. To fully evaluate the superiority of the method of the present application, four comparison models are set up:

[0150] DQN original model: state space only contains numerical state ; network structure: 3-layer fully connected neural network, learning rate 0.001, batch size 32;

[0151] DQN-GNN basic model: state space contains numerical state + graph structure data; GNN architecture: 2-layer graph convolution network; activation function: ReLU; joint training: GNN and DQN are updated synchronously;

[0152] DQN-GNN-unfrozen weights: GNN architecture is a 2-layer graph attention network; number of attention heads is 4; activation function is Tanh; additional layer is global average pooling + Dropout(0.1)

[0153] DQN-GNN freeze weight (the application): GNN pre-training is based on historical data for supervised learning pre-training; the training strategy adopts fixed GAT layer weight, and only optimizes DQN parameters; feature fusion: GNN output and numerical state splicing.

[0154] Table 1 Comparison of voltage control accuracy indicators

[0155] Algorithm MSE_1 MSE_2 MSE_3 Average MSE Improvement Margin DQN Original 3.56809 1.88123 3.10267 2.85066 - DQN-GNN 0.43856 1.00241 0.58030 0.67376 76.4% DQN-GNN Unfrozen Weights 0.49595 0.62330 0.60110 0.57345 79.9% The Method of the Present Application 0.36879 0.62635 0.60847 0.53454 81.3%

[0156] After introducing GNN, the voltage control accuracy is significantly improved, and the MSE is reduced by about 80% on average; the method of the application performs most stably in three independent experiments, and the variance is the smallest. The freeze weight strategy effectively prevents overfitting and ensures generalization performance, as shown in Figure 2 , as shown in Figure 2 is an experimental effect comparison diagram of the virtual power plant voltage-reactive power coordination control method based on the fusion of graph neural network and deep reinforcement learning.

[0157] Table 2 Comparison of algorithm robustness indicators

[0158] Algorithm Var_1 Var_2 Var_3 Average VAR Stability DQN-Original 0.06655 0.07396 0.08969 0.07673 - DQN-GNN 0.04140 0.05604 0.07747 0.05830 24.0% DQN-GNN Unfrozen Weights 0.05964 0.06588 0.07158 0.06570 14.4% The Method of the Present Application 0.05288 0.06350 0.07657 0.06432 16.2%

[0159] The method of the application has excellent robustness while maintaining high accuracy. The freeze weight strategy effectively suppresses performance fluctuations during training.

[0160] Table 3 Voltage limit probability statistics

[0161] Algorithm Experiment 1 Out-of-limit Rate Experiment 2 Out-of-limit Rate Experiment 3 Out-of-limit Rate Average Out-of-limit Rate Safety DQN Original 37.50% 27.25% 32.45% 32.40% - DQN-GNN 10.80% 43.45% 5.10% 19.78% 39.0% DQN-GNN Unfrozen Weights 9.30% 4.20% 6.25% 6.58% 79.7% The Present Application 4.00% 4.00% 6.35% 4.78% 85.2%

[0162] The method of the application reduces the voltage limit probability from 23.32% to 4.78%, with an improvement of 85.2%. The method of the application can effectively coordinate virtual power plant resources and distribution network equipment, quickly adjust DG reactive power output when the voltage is limited, reduce the number of capacitor actions, and optimize network loss. Experiments show that the utilization rate of virtual power plant resources increases by 15%, and the overall operating cost decreases by 12%. It performs consistently in three independent experiments, proving its reliable voltage safety guarantee capability.

[0163] A virtual power plant voltage-reactive power coordination control system based on the fusion of graph neural network and deep reinforcement learning, the system comprises:

[0164] A data acquisition and graph construction unit configured to acquire distribution network data in real time and construct the topological graph;

[0165] A GNN feature extraction unit embedded with a pre-trained and weight-frozen GNN module, configured to receive the topological graph and output the global graph feature vector;

[0166] a state fusion unit, configured to concatenate the global graph feature vector and the system numerical state vector into the fused state vector;

[0167] a DQN decision unit, configured to receive the fused state vector, output Q values of each control action, and select an optimal action;

[0168] a control execution unit, configured to issue the optimal action to corresponding distributed resources and power distribution network devices;

[0169] an experience replay and learning unit, configured to store interaction experience, calculate rewards and sample data, and update parameters of the DQN decision unit.

[0170] Specifically, in the embodiment, the system integrates the five functions of perception, representation, decision, execution and learning through modular design, forming a closed-loop intelligent regulation architecture. The specific composition and functions are as follows:

[0171] 1. Data acquisition and graph construction unit

[0172] This unit is responsible for real-time acquisition of operating state data from data sources such as power distribution network SCADA system, PMU device, virtual power plant energy management system, etc., including bus voltage, line power, device state, distributed power output, energy storage SOC, etc. On this basis, the physical power grid is abstracted into a topology graph G=(V,E), where the node set V corresponds to the bus in the power grid, and the edge set E corresponds to the power line connecting the bus. Each node is assigned a node feature vector containing voltage amplitude, phase angle, active / reactive power injection, etc.; each edge is assigned an edge feature vector consisting of line impedance, real-time active / reactive power flow, etc. This unit outputs structured graph data, laying the foundation for subsequent topology perception.

[0173] 2. GNN feature extraction unit

[0174] This unit embeds a pre-trained and weight-frozen graph neural network for efficient extraction of high-order topological correlation features of the power grid. Its input is the topology graph constructed as described above, which aggregates neighbor information layer by layer through a multi-layer message passing mechanism, and finally generates an embedding representation for each node rich in k-order neighborhood context. Subsequently, a global readout function (such as global average pooling) is used to compress all node embeddings into a fixed-dimensional global graph feature vector. Since the GNN module has been pre-trained and the weights are frozen during the training phase, this unit only performs forward inference, with low computational overhead and fast response, suitable for online deployment, while retaining the deep representation ability of the electrical coupling relationship and overall operating situation of the power grid.

[0175] 3. State fusion unit

[0176] This unit is responsible for fusing the global graph feature vector from the GNN feature extraction unit with the system numerical state vector from the monitoring system. The latter includes but is not limited to: all controllable device states, node voltage / current, active / reactive power injection, line flow, OLTC tap ratio, capacitor switching state, energy storage SOC, and controllable load state, etc. Both are directly concatenated in the feature dimension to form a high-dimensional, information-complete fusion state vector. This vector not only contains the non-Euclidean topological structure information of the power grid, but also contains the system-level dynamic operation indicators, providing a comprehensive and accurate state representation for intelligent decision-making.

[0177] 4. DQN decision unit

[0178] This unit is composed of a deep Q network, whose core is a multi-layer fully connected feedforward neural network. It takes the fusion state vector as input, and after nonlinear transformation through several hidden layers, it outputs a corresponding Q value estimate for each executable action in the predefined joint action space at the output layer. The joint action space is composed of the action combination of virtual power plant internal resources (such as photovoltaic inverter reactive power tap, energy storage charge and discharge power) and distribution network equipment (such as OLTC tap adjustment, capacitor bank switching). DQN selects the action with the maximum Q value based on the ε-greedy strategy as the optimal coordinated control action, achieving a balance between exploration and utilization, and possessing end-to-end policy mapping capability.

[0179] 5. Control execution unit

[0180] This unit receives the optimal action code output by the DQN decision unit and parses it into specific device control instructions. For example, "action ID = 27" is decoded as "OLTC up 1 tap + turn on the 2nd group of capacitors + energy storage discharges at 0.4 p.u.". Then, through standard communication protocols (such as IEC 61850, Modbus, or DNP3), the instructions are synchronously issued to the distributed resource controllers (such as inverters, energy storage converters, controllable load terminals) inside the virtual power plant and the distribution network automation system (such as distribution terminals, capacitor controllers). This unit ensures the accurate and timely execution of control instructions, completing a closed-loop regulation operation.

[0181] 6. Experience replay and learning unit

[0182] The unit is the key to realize continuous learning and strategy evolution of the system. After each control cycle, it encapsulates the interaction data as experience quadruple and stores it in the experience replay buffer. The immediate reward is calculated according to the preset multi-objective reward function (comprehensive voltage deviation, network loss, device action cost, VPP operation cost, etc.). When the buffer samples are sufficient, the unit randomly samples a batch of experiences, calculates the stable learning target by using the target network, and updates the network parameters of the DQN decision unit by minimizing the Bellman error. By periodically synchronizing the main network and target network parameters, the training stability is ensured. The unit supports offline or online fine-tuning, enabling the system to adapt to changes in power grid structure, load fluctuations and new energy uncertainty, and ultimately converging to a stable and efficient voltage-reactive power coordination control strategy.

[0183] It should be noted that in this text, the term "comprise", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or system. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of another identical element in the process, method, article or system that includes the element.

[0184] The above sequence numbers of the embodiments of the present application are only for description, not representing the advantages and disadvantages of the embodiments.

[0185] It should be particularly noted that, through the description of the above embodiments, those skilled in the art can clearly understand the embodiments, which can be implemented by means of software and necessary general hardware platform, and of course can also be implemented by hardware. Based on such understanding, the above technical solutions essentially or say the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a computer readable storage medium such as ROM / RAM, magnetic disk, optical disk, etc., including a plurality of instructions to make a computer device (which can be a personal computer, server, or network device, etc.) execute the method described in each embodiment or some part of the embodiment.

[0186] Finally, it should be pointed out that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A virtual power plant voltage-reactive power coordination control method based on the fusion of graph neural networks and deep reinforcement learning, characterized in that, The method comprises the following steps: real-time collection of operation state data of the power distribution network, modeling of the power distribution network as a topology graph, wherein nodes represent busbars and edges represent lines, and each node and edge is respectively assigned a node feature vector and an edge feature vector containing real-time measurement data; inputting the topology graph into a pre-trained GNN module and generating a global graph feature vector fused with topology information; concatenating the global graph feature vector with a system numerical state vector not containing topology information to form a fusion state vector; inputting the fusion state vector into a deep network DQN, and outputting, by the DQN, a Q value for coordinated control of internal distributed resources of the virtual power plant and power distribution network equipment; selecting an optimal coordinated control action according to the Q value and performing the action, and obtaining an instant reward according to a preset multi-objective reward function; storing a four-tuple data composed of a current state, a selected action, an instant reward and a next state in an experience replay buffer, and updating network parameters of the DQN by using data in the experience replay buffer, and repeating the above steps until the DQN converges, so that a stable coordinated control strategy of the virtual power plant is obtained for coordinated control of internal distributed resources of the virtual power plant and power distribution network equipment.

2. The method of claim 1, wherein, The step of real-time collection of operation state data of the power distribution network, modeling of the power distribution network as a topology graph, wherein nodes represent busbars and edges represent lines, and each node and edge is respectively assigned a node feature vector and an edge feature vector containing real-time measurement data, comprises: defining a power grid topology structure, identifying busbars and lines in the power distribution network, and constructing a topology graph G=(V,E) representing the structure of the power grid, wherein V represents a node set in the power grid, E represents an edge set in the power grid, each node vi∈V corresponds to a busbar in the power grid, and each edge eij∈E represents a power line connecting two busbars; the feature vector of each node vi contains real-time measurement data related to the busbar, such as voltage amplitude, phase angle, injected active power and reactive power; the feature vector of each edge eij describes the line properties and real-time operating parameters of the line connecting two busbars, such as line impedance, active power and reactive power.

3. The method of claim 1, wherein, The step of inputting the topology graph into a pre-trained GNN module and generating a global graph feature vector fused with power grid topology correlation information comprises: inputting the constructed power grid topology graph into a pre-trained GNN module, the GNN module iteratively aggregates feature information of each node and its neighbor nodes through an L-layer message passing mechanism, in each layer, the embedding vector of each node is updated according to its own embedding in the previous layer and the embedding of its adjacent nodes, thereby capturing topology context information from local to global layer by layer, and finally generating an L-layer node embedding vector for each node rich in k-order neighborhood topology correlation information; applying a global readout function to the set of all L-layer node embedding vectors output by the GNN module, the global readout function being used to aggregate a variable number of node embeddings into a fixed-dimensional vector; Through the aggregation operation of the global readout function, a global graph feature vector capable of representing the entire power grid topology and its dynamic interaction is generated, which is the final output fused with the power grid topology information.

4. The method of claim 1, wherein, The step of splicing the global graph feature vector and the system numerical state vector not containing topology information to form a fusion state vector comprises: obtaining a global graph feature vector output by the graph neural network module, the vector being a fixed-dimensional embedding obtained through global pooling operation after multi-layer message passing and aggregation on the distribution network topology graph, and containing the electrical coupling relationship between nodes and the overall operation situation; synchronously collecting a group of system numerical state vectors not directly dependent on the power grid topology structure; directly splicing the global graph feature vector and the system numerical state vector in the feature dimension to constitute a high-dimensional fusion state vector.

5. The method of claim 4, wherein, The system numerical state vector includes but is not limited to the current state of all controllable devices, node voltage amplitude, node current injection, node active power injection, node reactive power injection, line active power flow, line reactive power flow, on-load tap changer ratio, capacitor bank switching state, distributed generator output power, energy storage system state of charge and controllable load state.

6. The method of claim 1, wherein, The step of inputting the fusion state vector into the deep network DQN and outputting the Q value of the virtual power plant for the coordination control of the internal distributed resources and distribution network equipment of the DQN comprises: taking the fusion state vector as the input of the deep network DQN, wherein the DQN is a feedforward neural network with multiple fully connected layers, and the network parameters of the DQN are θ; receiving the fusion state vector through the input layer of the DQN and sequentially performing nonlinear feature transformation through at least one hidden layer; outputting a corresponding Q value estimate for each executable action in the action space at the output layer of the DQN.

7. The method of claim 1, wherein, The step of selecting the optimal coordination control action according to the Q value and executing it while obtaining the immediate reward according to the preset multi-objective reward function comprises: based on the Q value corresponding to each action output by the DQN, using the ε-greedy strategy to select the action, wherein with a probability of 1−ε, the action with the maximum Q value is selected as the current optimal coordination control action, and with a probability of ε, a random action is selected to maintain the exploration ability; analyzing the selected current optimal coordination control action into specific device control instructions and synchronously issuing them to the distributed resources inside the virtual power plant and the devices in the distribution network to complete a coordination control operation; after the execution of the optimal coordination control action, collecting the running state data of the next moment from the distributed resources inside the virtual power plant and the devices in the distribution network; based on the running state data of the next moment, calculating the immediate reward value according to the preset multi-objective reward function.

8. The method of claim 1, wherein, The multi-objective reward function r_t is designed as r_t=-J_t, wherein J_t is a comprehensive cost function at time t, and the comprehensive cost function J_t includes one or more of the following weighted sums: voltage deviation penalty term, power factor correction term, network loss penalty term, device action frequency penalty term and virtual power plant operation cost term.

9. The method of claim 1, wherein, The four-tuple data composed of the current state, the selected action, the immediate reward and the next state is stored in an experience replay buffer, and the network parameters of the DQN are updated using the data in the experience replay buffer, and the above steps are repeated until the DQN converges, and a stable virtual power plant coordination control strategy is obtained, and the step of coordinating and controlling the internal distributed resources and power distribution network equipment of the virtual power plant comprises: After each coordination control action cycle ends, the current fusion state vector st, the selected optimal coordination control action at, the calculated immediate reward rt, and the next fusion state vector st+1 observed after the action are packaged into an experience four-tuple (st, at, rt, st+1), and stored in an experience replay buffer with a preset capacity; When the number of samples in the experience replay buffer reaches a preset minimum training threshold, the update process of the DQN network parameters is started; When the loss function of the DQN network converges to a stable level, and the performance index in the verification scene reaches a preset threshold and remains stable, it is determined that the training is completed, and the strategy generated by the DQN finally obtains a stable virtual power plant voltage-reactive power coordination control strategy.

10. A virtual power plant voltage-reactive power coordination control system based on the fusion of graph neural networks and deep reinforcement learning, characterized in that, The system comprises: A data acquisition and graph construction unit configured to acquire power distribution network data in real time and construct the topology graph; A GNN feature extraction unit embedded with a pre-trained and weight-frozen GNN module, configured to receive the topology graph and output the global graph feature vector; A state fusion unit configured to concatenate the global graph feature vector and the system numerical state vector into the fusion state vector; A DQN decision unit configured to receive the fusion state vector, output the Q value of each control action, and select the optimal action; A control execution unit configured to issue the optimal action to the corresponding distributed resources and power distribution network equipment; An experience replay and learning unit configured to store interaction experience, calculate rewards and sample data, and update the parameters of the DQN decision unit.

Images