A voltage and reactive power control method and system for a microgrid

Abstract

The application discloses a voltage reactive power control method and system of a micro-grid. It belongs to the field of micro-grid control. The application obtains the power distribution network topology, the distributed energy node device parameters and the real-time operation data, configures intelligent agents and constraint boundaries for each node. The real-time data is input into a lightweight decision model based on multi-agent deterministic policy gradient. The model combines a local priority experience replay mechanism and a differentiable projection module, and after asynchronous gradient aggregation, structured pruning and low-bit-width quantization processing, it outputs continuous reactive power control instructions that meet the voltage stability and reactive power balance constraints, realizing closed-loop control of the distributed energy node devices. The application effectively solves the problem that the prior art cannot accurately and efficiently control the voltage of the micro-grid.

Description

Technical Field

[0001] This invention relates to the field of microgrid control, and more particularly to a voltage and reactive power control method and system for microgrids. Background Technology

[0002] With the high penetration of distributed photovoltaic, wind power, and energy storage into microgrids, the system is gradually exhibiting significant characteristics such as strong volatility, weak inertia, and topological time-varying nature, posing severe and complex challenges to voltage and reactive power control. Traditional voltage and reactive power control methods mostly rely on centralized optimization or rule-driven modes. These methods are difficult to adapt to the dynamic and decentralized operating environment of microgrids, not only having slow response speeds but also lacking flexible adaptive capabilities, and can no longer meet the control requirements in scenarios with a high proportion of distributed energy resources.

[0003] In recent years, multi-agent reinforcement learning, with its advantages in distributed collaborative decision-making, has emerged as a potential solution to the aforementioned challenges. Among them, methods based on deterministic policy gradients have shown great potential in the field of voltage and reactive power regulation in microgrids due to their ability to effectively handle continuous action space problems. However, existing technologies still face three core challenges in practical deployment and application: First, poor adaptability to computational resources. Existing policy networks often have large parameter scales, making it difficult to directly deploy them on resource-constrained embedded controllers such as photovoltaic inverters and energy storage converters. Second, low training efficiency. The time-varying and highly uncertain nature of microgrid operating states leads to significant differences in the value of experience samples collected by agents. A large number of low-information-density samples dilute key learning signals, resulting in slow policy convergence and difficulty in forming reliable control capabilities within a limited training window. Third, lack of physical constraints. The generated reactive power regulation commands do not explicitly consider physical constraints such as equipment capacity and voltage safety limits, posing execution risks and failing to comply with the core operating principle of "safety first" in power systems. This severely restricts the engineering implementation and widespread application of the technology. These shortcomings prevent existing technologies from accurately and efficiently controlling the voltage of microgrids. Summary of the Invention

[0004] This invention provides a voltage and reactive power control method and system for microgrids, to solve the problem that existing technologies cannot accurately and efficiently control the voltage of microgrids.

[0005] In a first aspect, this application provides a voltage and reactive power control method for a microgrid, comprising: Acquire the distribution network topology, distributed energy node equipment parameters, and real-time operating data of each node; Based on the aforementioned topology and distributed energy node device parameters, each intelligent agent and each constraint boundary are configured for each distributed energy node. Based on each intelligent agent and each constraint boundary, the real-time operating data of each node is input into a preset lightweight decision model so that the lightweight decision model outputs a continuous reactive power control command that satisfies the voltage stability margin and reactive power balance constraints. The lightweight decision model is based on multi-agent deterministic policy gradient, combined with a local priority experience replay mechanism, and is obtained by constraining and mapping control actions through a preset differentiable projection module, and then training through asynchronous gradient aggregation, structured pruning and low bit-width quantization. Closed-loop control is performed on the distributed energy node equipment of the distribution network according to the continuous reactive power control command.

[0006] This application first acquires the topology of the distribution network, the equipment parameters of distributed energy nodes, and the real-time operating data of each node. This data provides the foundation for subsequent agent configuration and decision-making. Based on this data, an agent is configured for each distributed energy node, and corresponding constraint boundaries are set to ensure that the decisions of each agent comply with the physical constraints of voltage stability and reactive power balance. Subsequently, the real-time operating data is input into a pre-defined lightweight decision model. This model is based on multi-agent deterministic policy gradients, combined with a local priority experience replay mechanism, and uses a differentiable projection module to perform constraint mapping on control actions. This process ensures that the generated control commands not only meet physical constraints but also efficiently utilize key experience data, improving learning efficiency. In addition, the model undergoes asynchronous gradient aggregation, structured pruning, and low-bit-width quantization during the training phase, significantly reducing model complexity and enabling efficient deployment on resource-constrained embedded devices. Finally, the lightweight decision model outputs continuous reactive power control commands that satisfy voltage stability margin and reactive power balance constraints, realizing closed-loop control of distributed energy node equipment in the distribution network. Through this design, this application effectively solves the problem that existing technologies cannot accurately and efficiently control the voltage of microgrids.

[0007] Furthermore, the configuration of each intelligent agent and each constraint boundary for each distributed energy node based on the topology and distributed energy node device parameters specifically includes: Based on the distribution network topology, the neighboring node association relationship of each distributed energy node is determined. Combining the installed capacity, controllable type and real-time output range in the equipment parameters of the distributed energy node, the voltage safety range and reactive power regulation range of each node are determined, and the constraint boundary corresponding to each intelligent agent is obtained. Each distributed energy node is configured with an independent intelligent agent, so that the distributed energy node device corresponds one-to-one with each intelligent agent, and each intelligent agent is only associated with the operating data and constraint boundaries of its own node and neighboring nodes.

[0008] This application analyzes the distribution network topology and distributed energy node equipment parameters to configure an independent intelligent agent for each distributed energy node and determine its constraint boundaries. Specifically, firstly, based on the distribution network topology, the association relationships between neighboring nodes of each distributed energy node are determined. Then, combining the installed capacity, controllable type, and real-time output range of each node, the voltage safety range and reactive power regulation capability range of each node are calculated, thereby obtaining the constraint boundaries corresponding to each intelligent agent. This configuration method allows each intelligent agent to associate only with the operating data and constraint boundaries of its own node and neighboring nodes, realizing distributed collaborative control. In this way, the present invention can effectively solve the problems of lack of safety constraints in control commands and difficulty in adapting to the high penetration rate of distributed energy in the prior art, significantly improving the voltage stability and reactive power resource utilization efficiency of microgrids under complex operating conditions, while reducing computational complexity, improving the real-time performance and adaptive capability of the system, and providing an efficient and reliable solution for the intelligent operation of new power systems.

[0009] Furthermore, based on each intelligent agent and each constraint boundary, the real-time operating data of each node is input into a preset lightweight decision model, so that the lightweight decision model outputs a continuous reactive power control command that satisfies the voltage stability margin and reactive power balance constraints, specifically as follows: The real-time running data of each intelligent agent and its corresponding node are combined to obtain each first intelligent agent. Each first intelligent agent is then input into the built-in policy network of the lightweight decision model to generate the original control action. By using the differentiable projection module integrated in the lightweight decision model, and combining the constraint boundaries corresponding to each first agent, the original control actions are mapped to obtain feasible control actions that meet the constraints of voltage stability margin and reactive power balance. The feasible control actions of each first agent are summarized and continuous reactive power control instructions are output.

[0010] This application combines real-time operational data from each distributed energy node with corresponding intelligent agents to form a first intelligent agent with real-time perception capabilities, which is then input into a lightweight decision model. The model first generates initial control actions through a built-in policy network, and then uses a differentiable projection module to map these actions to the constraint boundaries of each intelligent agent, ensuring that the generated control actions meet voltage stability margin and reactive power balance constraints. Finally, feasible control actions from each intelligent agent are aggregated, and continuous reactive power control commands are output. This process, through the efficient processing of the lightweight decision model, ensures the safety and feasibility of the control commands, while significantly improving the voltage stability and reactive power resource utilization efficiency of the microgrid under complex operating conditions. It achieves low-latency, high-security, and self-coordinated voltage and reactive power closed-loop control, providing an efficient and reliable solution for the intelligent operation of new power systems.

[0011] Furthermore, the original control actions are mapped using the differentiable projection module integrated in the lightweight decision model, combined with the constraint boundaries corresponding to each first agent, to obtain feasible control actions that meet the voltage stability margin and reactive power balance constraints. Specifically: This enables the differentiable projection module to construct a set of linear inequality constraints based on a preset voltage safety range and reactive power regulation capability range; Based on the original control actions generated by each first intelligent agent, within the range of the linear inequality constraint set, the feasible solution closest to the original control actions is found by solving a quadratic programming problem; The feasible solution is taken as a feasible control action that meets the constraints of voltage stability margin and reactive power balance.

[0012] This application utilizes a differentiable projection module within a lightweight decision-making model, combined with a pre-defined voltage safety range and reactive power regulation capability range, to construct a set of linear inequality constraints. For each initial control action generated by the first agent, the differentiable projection module finds the feasible solution closest to the initial control action by solving a quadratic programming problem within the constraint set. This process ensures that the control action, while satisfying voltage stability margin and reactive power balance constraints, approximates the ideal initial action as closely as possible, thereby improving the efficiency and adaptability of control commands while ensuring the safe operation of the power system. This design effectively solves the problem of control commands potentially violating physical constraints in existing technologies, significantly improving the voltage stability and reactive power resource utilization efficiency of microgrids under complex operating conditions.

[0013] Furthermore, the lightweight decision-making model is based on multi-agent deterministic policy gradients, combined with a local-first experience replay mechanism. It uses a pre-defined differentiable projection module to constrain and map control actions, and is trained through asynchronous gradient aggregation, structured pruning, and low-bit-width quantization. Specifically: Based on the distribution network topology and the equipment parameters of distributed energy nodes, each intelligent agent is configured for each distributed energy node, and the policy network and value network of each intelligent agent are initialized. This enables each intelligent agent to generate original control actions based on local real-time operating data, and to map the original control actions to a feasible range that meets the constraints of voltage stability margin and reactive power balance through a preset differentiable projection module, thereby obtaining constrained feasible actions. The action is applied to the corresponding distributed energy node device. After a control interaction is completed, the latest local observation data and reward value after the interaction are obtained. Combined with the initial local observation data before the control interaction and the action, an experience data set is constructed. After initializing the experience priority of the experience data in the experience data group according to the preset rules, it is stored in the local priority experience playback buffer. Sampling with replacement is performed on the experience data from the local priority experience replay buffer according to the experience priority, and the policy gradient and value gradient are calculated based on the sampled experience data. Based on the policy gradient and value gradient, and combined with asynchronous gradient aggregation, structured pruning, and low-bit-width quantization, the lightweight decision model is trained.

[0014] This application's lightweight decision-making model achieves efficient and secure voltage and reactive power control in microgrids through a series of innovative technologies. First, based on the distribution network topology and the device parameters of distributed energy nodes, an agent is configured for each node, and the policy network and value network are initialized. The agent generates initial control actions based on local real-time operating data and maps them to a feasible range that satisfies voltage stability and reactive power balance constraints through a differentiable projection module, thereby ensuring the safety and effectiveness of the control actions. Subsequently, the feasible actions are applied to the distributed energy node devices, acquiring the observed data and reward values ​​after interaction, and constructing an experience data set by combining the initial observation data. By initializing experience priorities and storing them in a local priority experience replay buffer, the model can sample experience data according to priority and calculate the policy gradient and value gradient. Finally, by combining asynchronous gradient aggregation, structured pruning, and low-bit-width quantization, a lightweight decision-making model is trained. This process not only improves the model's training efficiency and adaptability but also significantly reduces model complexity, enabling efficient deployment on resource-constrained embedded devices. This provides low-latency, high-security, and self-cooperative voltage and reactive power closed-loop control capabilities for the intelligent operation of microgrids.

[0015] Furthermore, the lightweight decision model is trained based on the policy gradient and value gradient, combined with asynchronous gradient aggregation, structured pruning, and low-bit-width quantization, specifically as follows: When the preset synchronization condition is met, based on the policy gradient and value gradient, the preset historical gradient momentum term is updated using the moving average method, and the parameters of the policy network and value network are adjusted and updated using the preset learning rate. The asynchronous gradients are aggregated to obtain the first parameter. Channel-level structured pruning is performed on the first parameter, and the norms of the output channel weights of each network layer of the policy network and the value network are calculated as each importance score. Sort all output channels by importance scores from low to high, select the output channel with the lowest importance score according to a preset ratio, and remove the input channels corresponding to the output channel with the lowest importance score. Perform structured pruning on the policy network and value network. Based on the symmetric quantization method, the scaling factor is calculated by combining the maximum absolute value of the weights of the network parameters of the pruned policy network and value network. The pruned network parameters are quantized to the range of signed integers using the scaling factor to generate lightweight model parameters. The lightweight model is trained based on the parameters of the lightweight model.

[0016] This application trains a lightweight decision-making model through a series of optimization techniques, significantly improving the model's efficiency and applicability. First, under preset synchronization conditions, based on the policy gradient and value gradient, the historical gradient momentum term is updated using a moving average method. Combined with a preset learning rate, the parameters of the policy network and value network are adjusted to complete asynchronous gradient aggregation, obtaining the initially optimized first parameters. Subsequently, channel-level structured pruning is performed on the first parameters. The L2 norm of the output channel weights of each network layer is calculated as an importance score, and the channels with the lowest importance and their corresponding input channel connections are removed after sorting by score, thus simplifying the network structure. Further, a symmetric quantization method is used to calculate a scaling factor based on the maximum absolute value of the pruned network parameter weights, quantizing the network parameters to the range of signed integers to generate lightweight model parameters. This process not only reduces the model's computational complexity and storage requirements but also ensures efficient deployment and operation of the model on resource-constrained embedded devices, ultimately achieving low-latency, high-security, and self-cooperative voltage and reactive power closed-loop control.

[0017] Secondly, this application provides a voltage and reactive power control system for a microgrid. The voltage and reactive power control system for the microgrid includes: The acquisition module is used to acquire the distribution network topology, distributed energy node equipment parameters, and real-time operating data of each node. The configuration module is used to configure each intelligent agent and each constraint boundary for each distributed energy node based on the topology and distributed energy node device parameters. The processing module is used to input the real-time running data of each node into a preset lightweight decision model based on each intelligent agent and each constraint boundary, so that the lightweight decision model outputs a continuous reactive power control command that satisfies the voltage stability margin and reactive power balance constraints. The lightweight decision model is based on multi-agent deterministic policy gradient, combined with a local priority experience replay mechanism, and is obtained by constraining and mapping control actions through a preset differentiable projection module, and then training through asynchronous gradient aggregation, structured pruning and low bit-width quantization. The control module is used to perform closed-loop control of the distributed energy node equipment in the distribution network according to the continuous reactive power control command.

[0018] The microgrid voltage and reactive power control system of this application acquires the topology of the distribution network, the parameters of distributed energy node devices, and the real-time operating data of each node through an acquisition module, providing basic data support for subsequent agent configuration and control command generation. The configuration module configures an agent and its constraint boundaries for each distributed energy node based on this data, ensuring that control decisions conform to physical constraints. The processing module inputs the real-time operating data into a lightweight decision model, which is based on multi-agent deterministic policy gradients, combined with a local-first experience replay mechanism. It performs constraint mapping on control actions through a differentiable projection module and optimizes the training through asynchronous gradient aggregation, structured pruning, and low-bit-width quantization. Finally, the model outputs continuous reactive power control commands that satisfy voltage stability margin and reactive power balance constraints. The control module performs closed-loop control of the distributed energy node devices according to these commands. This process not only improves the security and reliability of control commands, but also significantly reduces computational complexity and storage requirements through a lightweight model, enabling efficient deployment on resource-constrained embedded devices. This achieves low-latency, high-security, and self-cooperative voltage and reactive power closed-loop regulation of the microgrid under complex operating conditions, significantly improving the system's voltage stability and reactive power resource utilization efficiency. Attached Figure Description

[0019] Figure 1 : A schematic flowchart of an embodiment of the voltage and reactive power control method for a microgrid provided in this application; Figure 2 : A schematic diagram of an embodiment of lightweight autonomous decision-making parallel computation of multi-agent deterministic policy gradient provided in this application; Figure 3 : This is a schematic diagram of an embodiment of the voltage and reactive power control system for a microgrid provided in this application. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1 Please refer to Figure 1 In order to solve the problem that existing technologies cannot accurately and efficiently control the voltage of microgrids, this invention provides a voltage and reactive power control method for microgrids.

[0022] like Figure 2 As shown, Figure 2This is a schematic diagram illustrating the hierarchical architecture of the microgrid voltage and reactive power control method provided in this application embodiment. The method is divided into three stages: modeling, training, and deployment, corresponding to a multi-agent modeling unit, training functional units (local experience interaction and storage unit, asynchronous gradient calculation and uploading unit, and central coordination unit), and deployment functional units (embedded inference and online decision-making unit). These units collaborate to achieve autonomous regulation of voltage and reactive power. The following details the application process of this method, including steps S01-S04: S01: Obtain the distribution network topology, distributed energy node equipment parameters, and real-time operating data of each node.

[0023] In a preferred embodiment of this invention, the acquisition of the distribution network topology, distributed energy node equipment parameters, and real-time operating data of each node specifically includes: In the initialization and operation phases of this embodiment, three types of core data need to be comprehensively collected to support the full-process implementation of microgrid voltage and reactive power coordinated control. For the distribution network topology, by accessing the power grid dispatch management system, distribution network geographic information system (GIS), and equipment ledger database, the network architecture type (such as radial type, ring network type), node connection relationships (clarifying the interconnection logic of each bus, line, and distributed energy node), branch impedance parameters (including resistance and reactance values), switchgear status (opening and closing status), and key transformer parameters (turns ratio, rated capacity, short-circuit impedance) are extracted to accurately construct the physical connection and electrical characteristic model of the distribution network, providing a basic framework for the partitioned deployment of intelligent agents and the calculation of constraint boundaries.

[0024] For distributed energy node equipment parameters, the installation location (latitude and longitude, access bus number), installed capacity (active and reactive rated capacity), controllable type (such as photovoltaic inverter, static var generator (SVG), energy storage converter, etc.) and real-time output range (maximum / minimum adjustable reactive power, active power adjustment limit) of each node are obtained through equipment factory technical document filing, on-site commissioning and testing, and local controller data collection of equipment such as energy storage / photovoltaic inverters. This clarifies the controllability boundary of each node equipment and provides data support for subsequent constraint setting and control action feasibility verification.

[0025] Real-time operational data from each node is collected in seconds via embedded acquisition devices and voltage / current sensors deployed in distributed energy nodes, distribution network branches, and buses. This data includes the real-time voltage amplitude, active power (inflow / outflow direction, distinguished by positive / negative values), and reactive power (capacitive / inductive output status) of each distributed energy node. Simultaneously, voltage data from neighboring nodes is received via local communication networks (such as LoRa or Ethernet), and the real-time voltage of the nearest distributed energy node is selected to form the local observation dataset for each agent. This real-time data is continuously input into a lightweight decision-making model, providing dynamic input for the generation of initial control actions, environmental interaction feedback, and closed-loop control verification, ensuring precise matching between control decisions and the real-time operating status of the power grid.

[0026] S02: Based on the topology and distributed energy node device parameters, configure each intelligent agent and each constraint boundary for each distributed energy node.

[0027] As a preferred embodiment of this invention, the step of configuring each intelligent agent and each constraint boundary for each distributed energy node based on the topology and distributed energy node device parameters specifically includes: Based on the collected distribution network topology and distributed energy resource (DER) node device parameters, the precise configuration of intelligent agents is first completed: according to the node connection relationships, branch impedance distribution, and node location associations in the topology, the power supply range and coupling relationship of each DER node are clarified, ensuring that each DER node with reactive power regulation capability is configured with an independent intelligent agent, forming a one-to-one distributed control architecture of "node-intelligent agent". This allows each intelligent agent to focus on the voltage and reactive power status of its local node and associated area for autonomous decision-making, avoiding redundant cross-node communication and decision delays. More specifically, for each DER node with reactive power regulation capability, an independent intelligent agent is configured, forming a multi-agent set that corresponds one-to-one with physical devices, ensuring that control commands can directly act on local devices and support distributed collaborative control. Specifically, each independent intelligent agent is defined as follows: In the formula, Indexing for intelligent agents; For intelligent agent index set; The total number of intelligent agents is determined by the total number of DER nodes with reactive power regulation capabilities within the microgrid; Indicates the first in the system One intelligent agent; This represents the potential observation space of the intelligent agent, which consists of specific local observations; This represents the potential action space of the intelligent agent, which consists of specific, continuous control actions. This represents the policy network of the agent, which takes local observations as input and raw actions as output. It consists of a shallow MLP and has initialization parameters. ; This represents the value network of the intelligent agent, which, in order to save communication costs, uses local observation. With action Instead of joint observation and joint action as input, and scalar action value as output, it consists of a shallow MLP and has initial parameters. .

[0028] The local observations of each agent consist of local voltage amplitude, active power, reactive power, and the voltage of the nearest DER node, enabling it to make decisions based solely on locally measurable information. This also enhances the agent's perception of local grid coupling characteristics, thereby improving its decision-making effectiveness in voltage coordinated regulation. The specific definition of local observation is: In the formula, For time step index; For time step index set; This is an adjustable parameter, representing the maximum training time step; Indicates the first An intelligent agent in Local observation at a given moment; , , These are the voltage amplitude, active power, and reactive power of the DER node where the agent is located at the current moment, respectively. The power values ​​are signed floating-point numbers, with positive values ​​representing outflow power and negative values ​​representing inflow power. The index is the voltage of the nearest DER node at the current moment. This indicates the nearest neighbor node, determined by the power grid topology.

[0029] To achieve dynamic regulation of local voltage and stability of grid voltage, each agent adjusts the reactive power setpoint of its DER node at the current moment through continuous reactive power control actions, specifically defined as: In the formula, Indicates the first An intelligent agent in Actions that can be taken at any time; This is the reactive power setting value used for adjustment.

[0030] This step is the modeling process for a multi-agent system, completed during the system initialization phase, providing a model foundation for the subsequent local observation acquisition process of each agent.

[0031] Simultaneously, based on key information such as installed capacity, controllable type, real-time output range, and transformer ratio in the parameters of distributed energy node equipment, exclusive constraint boundaries are defined for each intelligent agent: On the one hand, by combining the nominal voltage reference value of the distribution network and the voltage tolerance threshold of the equipment, the voltage safety range of each node is set, and the upper and lower limits of voltage amplitude are clearly defined; on the other hand, based on the maximum adjustable reactive power capacity and output adjustment rate limit of the equipment, the reactive power adjustment capability range of the intelligent agent is determined, forming the boundary constraint of reactive power output; in addition, by combining the branch impedance characteristics and power flow distribution law in the topology, the sensitivity coefficient is calculated through power flow simulation to further refine the quantitative indicators of the constraint boundaries, ensuring that the constraint conditions not only meet the physical characteristics of the equipment but also adapt to the safety requirements of grid operation. The observation space and action space of each intelligent agent are also determined synchronously based on the above configuration: the observation space includes the voltage amplitude, active power, reactive power of the local node, and the voltage data of the nearest neighbor node; the action space is defined as the continuous adjustment range of the reactive power setpoint of the local distributed energy node, enabling the intelligent agent to make precise control decisions based on locally observable data within the clearly defined constraint boundaries.

[0032] S03: Based on each intelligent agent and each constraint boundary, the real-time running data of each node is input into a preset lightweight decision model so that the lightweight decision model outputs a continuous reactive power control command that satisfies the voltage stability margin and reactive power balance constraints. The lightweight decision model is based on multi-agent deterministic policy gradient, combined with a local priority experience replay mechanism, and uses a preset differentiable projection module to constrain and map control actions. It is then trained through asynchronous gradient aggregation, structured pruning, and low-bit-width quantization.

[0033] In a preferred embodiment of this invention, based on each intelligent agent and each constraint boundary, the real-time operating data of each node is input into a preset lightweight decision model, so that the lightweight decision model outputs a continuous reactive power control command that satisfies the voltage stability margin and reactive power balance constraints. Specifically: Each distributed energy node's corresponding agent independently collects real-time operating data (including voltage amplitude, active power, reactive power, etc.) from its own node and neighboring nodes based on preset constraint boundaries (including voltage safety range and reactive power regulation range). This real-time operating data is then input into a lightweight decision model deployed on a locally embedded device. This lightweight decision model has a built-in policy network. First, it performs feature extraction and mapping processing on the input real-time operating data to generate initial control actions under unconstrained conditions. Then, through an integrated differentiable projection module, the model constructs a set of linear inequality constraints in conjunction with preset constraint boundaries. The initial control actions are substituted into this constraint set, and a quadratic programming problem is solved to find the feasible solution closest to the initial control actions. This completes the constraint verification and correction of the control actions, ensuring that the actions conform to both the physical regulation capabilities of the distributed energy equipment and the voltage stability margin requirements of the distribution network. Meanwhile, during the processing, the model implicitly matches the reactive power output of each node through multi-agent collaborative logic to ensure the reactive power balance of the entire microgrid. Finally, it outputs continuously adjustable reactive power control commands. The entire command generation process is completed independently on the embedded device without cross-node communication. The response delay is controlled within 10 milliseconds, ensuring the real-time performance and autonomy of regulation.

[0034] The lightweight decision model is based on multi-agent deterministic policy gradients, combined with a local-first experience replay mechanism. It uses a pre-defined differentiable projection module to constrain control actions, and is trained through asynchronous gradient aggregation, structured pruning, and low-bit-width quantization. Specifically: The training process of the lightweight decision-making model is based on the multi-agent deterministic policy gradient as the core framework. It achieves efficient training and lightweight deployment adaptation through multi-module collaborative optimization. The specific process is as follows: Each agent corresponding to the distributed energy node is configured with an independent policy network and a value network. The policy network generates the original control actions in the continuous domain with local real-time observation data as input, while the value network is used to evaluate the state value after the action is executed, providing gradient guidance for policy updates.

[0035] Each agent acquires the voltage amplitude of its current DER node in real time, based on a predefined local observation dimension. Active power reactive power It also transmits the voltage amplitude of its current node via local broadcast, and uses the voltage amplitude of the nearest neighboring DER node received first as the current voltage of the nearest neighboring DER node. Ultimately, complete local observations were obtained. .

[0036] This complete local observation provides input support for the generation of the original action, based on the local observation at this current moment. Each agent uses this as input and passes it to its own policy network to generate the original action. The specific process is as follows: In the formula, This indicates the original action, which represents an unconstrained reactive power regulation command under ideal conditions, but may exceed the equipment's capacity or cause voltage over-limits.

[0037] Subsequently, each agent will perform the original action. The possible actions at the current moment are generated by inputting the self-differentiable projection module. The possible actions are obtained by solving the following quadratic programming problem: In the formula, For projection operations, it means from the set Find the distance parameter Recent projection; Indicates that the function Parameters that obtain the minimum value The possible values ​​of ; express Norm; The actions that each agent can take at the current moment; Set the actionable actions for each agent at the current moment. To concentrate actions for each action that is possible.

[0038] The set of possible actions is defined by linear inequality constraints: In the formula, This is the constraint matrix for the intelligent agent; This is the constraint vector for the agent; The sensitivity coefficient is determined by the network topology and impedance, and can be obtained from power flow simulation during the system initialization phase. and These are the minimum and maximum values ​​of the system voltage safety range, respectively. and These are the minimum and maximum adjustable reactive power values ​​for the current DER device, respectively.

[0039] According to the implicit function theorem, the above differentiable projection module supports gradient backpropagation, and its gradient transfer matrix is: In the formula, The gradient transfer matrix; and These are respectively movable actions and primitive actions; It is the identity matrix; Constraint matrix in projection operation The active constraint set; Constraint matrix The active constraint corresponds to the submatrix formed by the row vectors, which ensures that the gradient of the projection process can be backpropagated, thus guaranteeing the convergence of model training.

[0040] Based on the generated current actionable actions Each agent adjusts the reactive power setpoint of its local DER device in real time. After one control cycle, it acquires the latest local observation data and calculates the reward for the current moment. The reward calculation formula is as follows: In the formula, The reward for the agent at the current moment; , , These are non-negative weighting coefficients, representing the relative importance of voltage deviation penalty, reactive power usage cost, and jitter penalty, respectively. This represents the voltage amplitude of the node where the agent is located at the next moment. This is the reference value for the nominal voltage of the distribution network; The reactive power of the node where the agent is located at the next moment; and These are the possible actions for the current moment and the previous moment, respectively.

[0041] Each agent combines current local observation data and actionable actions Reward Value and the latest local observation data This forms a complete set of experience data, which is then stored in a local priority experience replay buffer. This buffer is a bounded storage structure maintained locally by the agent, with a maximum capacity of [number of experience records]. Determined by the embedded resources of the DER node where the agent resides, when the buffer capacity reaches its limit, a first-in, first-out (FIFO) strategy is used to remove old experiences to maintain storage efficiency and sample diversity. Each newly stored experience has an initial priority, specifically: In the formula, Representing experience Corresponding priority; For buffer The highest priority already stored in the database.

[0042] At the same time, the sampling probability of each experience is calculated. The specific calculation process is as follows: In the formula, Representing experience The corresponding sampling probability; For buffer Number of experiences in the China General Chamber of Commerce; For buffer The sum of all empirical priorities is used to provide a basis for subsequent priority-based sampling. During training, a local priority experience replay mechanism is employed to improve sample utilization efficiency: the agent periodically samples with replacement from the buffer according to empirical priorities. Based on the sampled empirical data and corresponding sampling probabilities, the policy objective function and value objective function are estimated. The specific calculation process is as follows: In the formula, and These are the parameters of the policy network and the value network at the current moment, respectively. express The square of the norm; and Let these represent the strategy objective function and the value objective function, respectively. This is the discount factor.

[0043] Based on the time-series difference error obtained during the calculation of the value function, the priority of each experience is dynamically updated. The specific update rules are as follows: In the formula, To prevent regular expressions with a priority of zero.

[0044] Meanwhile, the deterministic policy gradient and value gradient are estimated based on sampling experience. The specific calculation process is as follows: In the formula, For gradient operators, express right The gradient; and These are the deterministic policy gradient and the value gradient, respectively. The gradient transfer matrix of the projection operation; The set of actions that each agent can take at the current moment.

[0045] When preset time-triggered conditions or gradient change conditions are met, each agent uploads its calculated policy gradient and value gradient to the central coordination unit. The time-triggered condition is when the local training steps reach a preset synchronization period. The gradient change is triggered when the norm of the current deterministic policy gradient exceeds a dynamic threshold or the norm of the value gradient exceeds a dynamic threshold. The specific calculation process for the dynamic threshold is as follows: In the formula, This is the sensitivity coefficient. Indicates the recent The gradient L2 norm of the training step is taken as a moving average.

[0046] When the agent meets the preset synchronization conditions, it uploads the calculated gradient data to the central coordination unit. This unit uses a momentum mechanism to update the locally maintained historical gradients. The specific update process is as follows: In the formula, and These represent the momentum terms at the current moment and the previous moment, respectively. This is the momentum decay coefficient, used to control the smoothness of the historical gradient; This represents the policy gradient or value gradient uploaded by each agent at the current moment; the initial value of the momentum term. .

[0047] Based on the updated gradient momentum term, the central coordination unit further updates the parameters of the global policy and the value network. The specific update process is as follows: In the formula, and These are the unpruned and unquantized policy network and value network parameters for the next time step; and These are the parameters of the policy network and value network at the current time, before pruning and quantization. The learning rate is used to adjust the model update rate. and These are the gradient momentum terms for the policy network and the value network, respectively.

[0048] After parameter aggregation, the central coordination unit performs channel-level structured pruning and symmetric quantization on the global network. The overall expression is: In the formula, Indicates quantization operation. This indicates a pruning operation.

[0049] The pruning operation is performed as follows: For each layer of the network, the L2 norm of the weights of each output channel is calculated as an importance score, and the layer with the lowest importance score is selected. Each channel is pruned. The specific calculation process for the importance score is as follows: In the formula, express Norm; superscript This is the hidden layer index, representing the index of the hidden layer in the network. One hidden layer; subscript For channel indexing; For channel index set; Indicates the first Maximum number of output channels per hidden layer; Indicates the network number In the hidden layer, the th Weight matrix for each output channel.

[0050] At the same time, in order to maintain network connectivity, the network's first... The corresponding input channels of the layer are clipped, and the specific input channel index to be clipped is related to the first layer. The output channel indices of the layers are the same.

[0051] The quantization operation employs a symmetric quantization method, with its scaling factor calculated based on the maximum absolute value of the weights. The quantization result is truncated to the signed value. The range of representation for bit-integers, and the specific quantization formula are as follows: In the formula, Indicates quantization operation. For the weights to be quantified, Indicates that the parameter Limited to the range Inside, This indicates the rounding operation. This is the scaling factor.

[0052] After completing structured pruning and quantization operations, the central coordination unit broadcasts the compressed, lightweight global network parameters back to each agent for a new round of training. This continues until the training time step reaches the preset maximum step size. Finally, the optimal lightweight strategy network parameters are determined and output. .

[0053] The lightweight policy network parameters trained as described above The final deployment takes place at each Distributed Energy Resource (DER) node. During the deployment phase, each agent relies solely on its own locally collected observation data. (Local voltage amplitude, active power, reactive power, and voltage of the nearest DER node) serve as inputs to the lightweight policy network, which then generates the original actions through network inference. Then, the integrated differentiable projection module maps it into a final control action that conforms to physical constraints. The reactive power setpoint of the DER node is adjusted in real time. The entire inference and projection process is completed end-to-end on embedded devices such as photovoltaic inverters and energy storage converters, without the need to access the central coordination unit or other intelligent agents. The control latency is strictly controlled within 10 milliseconds. After deployment, each intelligent agent no longer performs training phase operations such as experience playback, gradient calculation and uploading, and parameter updates. It relies entirely on the locally fixed lightweight policy network and projection module to achieve autonomous, compliant, and real-time control decisions.

[0054] S04: Perform closed-loop control on the distributed energy node equipment of the distribution network according to the continuous reactive power control command.

[0055] In a preferred embodiment of this invention, the closed-loop control of the distributed energy node equipment in the distribution network according to the continuous reactive power control command specifically includes: After receiving the continuous reactive power control commands output by the lightweight decision model, the embedded control devices (such as photovoltaic inverters and energy storage converter controllers) of each distributed energy node immediately parse the core parameters such as reactive power setpoint and adjustment rate in the commands. Combined with the device's own operating status (such as current output and hardware limitations), they adjust the working parameters of the reactive power regulation module in real time. For photovoltaic inverters, the reactive power output ratio is changed by adjusting the PWM modulation strategy; for energy storage converters, the reactive power response amplitude is adjusted by optimizing the current loop control parameters to achieve precise execution of the commands.

[0056] After the control operation is completed, the embedded device, through its built-in voltage and current sensors and power metering module, collects real-time feedback data such as node voltage amplitude, actual reactive power output, and power factor within 10 milliseconds, and uploads it to the observation unit of the local intelligent agent. The intelligent agent compares the feedback data with the preset voltage safety range and reactive power balance threshold in real time to verify whether the control effect meets the constraints. If the feedback data shows that the node voltage exceeds the safety range or the reactive power is not in a balanced state, the intelligent agent immediately triggers the lightweight decision model to regenerate an adapted continuous reactive power control command and sends it to the embedded device again for control. If the feedback data meets the constraints, the current control parameters are maintained, and the monitoring and verification process of the next control cycle begins, forming a closed-loop control link of "command issuance—control execution—data feedback—effect verification—command optimization," ensuring that the distribution network voltage stability and reactive power balance continuously meet the standards. The entire closed-loop process does not rely on real-time intervention from the central node and is completed autonomously through the collaboration of the local device and the intelligent agent, ensuring the real-time performance and reliability of the control.

[0057] Example 2 Please refer to Figure 3 This is a voltage and reactive power control system for a microgrid provided in the embodiments of this application.

[0058] In this embodiment, the voltage and reactive power control system of the microgrid includes an acquisition module 10, a configuration module 20, a processing module 30, and a control module 40.

[0059] Module 10 is used to acquire the distribution network topology, distributed energy node equipment parameters, and real-time operating data of each node; Configuration module 20 is used to configure each intelligent agent and each constraint boundary for each distributed energy node based on the topology and distributed energy node device parameters; Processing module 30 is used to input the real-time running data of each node into a preset lightweight decision model based on each intelligent agent and each constraint boundary, so that the lightweight decision model outputs a continuous reactive power control command that satisfies the voltage stability margin and reactive power balance constraints. The lightweight decision model is based on multi-agent deterministic policy gradient, combined with a local priority experience replay mechanism, and is obtained by constraining and mapping control actions through a preset differentiable projection module, and then training through asynchronous gradient aggregation, structured pruning and low bit-width quantization. The control module 40 is used to perform closed-loop control on the distributed energy node equipment of the distribution network according to the continuous reactive power control command.

[0060] For ease of description and brevity, the system embodiments of the present invention include all the implementation methods in the above-described embodiments of the voltage and reactive power control method for microgrids, which will not be repeated here.

[0061] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A voltage and reactive power control method for a microgrid, characterized in that, include: Acquire the distribution network topology, distributed energy node equipment parameters, and real-time operating data of each node; Based on the aforementioned topology and distributed energy node device parameters, each intelligent agent and each constraint boundary are configured for each distributed energy node. Based on each intelligent agent and each constraint boundary, the real-time operating data of each node is input into a preset lightweight decision model so that the lightweight decision model outputs a continuous reactive power control command that satisfies the voltage stability margin and reactive power balance constraints. The lightweight decision model is based on multi-agent deterministic policy gradient, combined with a local priority experience replay mechanism, and is obtained by constraining and mapping control actions through a preset differentiable projection module, and then training through asynchronous gradient aggregation, structured pruning and low bit-width quantization. Closed-loop control is performed on the distributed energy node equipment of the distribution network according to the continuous reactive power control command.

2. The voltage and reactive power control method for a microgrid according to claim 1, characterized in that, The configuration of each intelligent agent and each constraint boundary for each distributed energy node based on the topology and distributed energy node device parameters is as follows: Based on the distribution network topology, the neighboring node association relationship of each distributed energy node is determined. Combining the installed capacity, controllable type and real-time output range in the equipment parameters of the distributed energy node, the voltage safety range and reactive power regulation range of each node are determined, and the constraint boundary corresponding to each intelligent agent is obtained. Each distributed energy node is configured with an independent intelligent agent, so that the distributed energy node device corresponds one-to-one with each intelligent agent, and each intelligent agent is only associated with the operating data and constraint boundaries of its own node and neighboring nodes.

3. The voltage and reactive power control method for a microgrid according to claim 1, characterized in that, Based on the various intelligent agents and constraint boundaries, the real-time operating data of each node is input into a preset lightweight decision model, so that the lightweight decision model outputs a continuous reactive power control command that satisfies the voltage stability margin and reactive power balance constraints. Specifically: The real-time running data of each intelligent agent and its corresponding node are combined to obtain each first intelligent agent. Each first intelligent agent is then input into the built-in policy network of the lightweight decision model to generate the original control action. By using the differentiable projection module integrated in the lightweight decision model, and combining the constraint boundaries corresponding to each first agent, the original control actions are mapped to obtain feasible control actions that meet the constraints of voltage stability margin and reactive power balance. The feasible control actions of each first agent are summarized and continuous reactive power control instructions are output.

4. The voltage and reactive power control method for a microgrid according to claim 3, characterized in that, The process involves mapping the original control actions using the differentiable projection module integrated in the lightweight decision model, combined with the constraint boundaries corresponding to each first agent, to obtain feasible control actions that meet the constraints of voltage stability margin and reactive power balance. Specifically: This enables the differentiable projection module to construct a set of linear inequality constraints based on a preset voltage safety range and reactive power regulation capability range; Based on the original control actions generated by each first intelligent agent, within the range of the linear inequality constraint set, the feasible solution closest to the original control actions is found by solving a quadratic programming problem; The feasible solution is taken as a feasible control action that meets the constraints of voltage stability margin and reactive power balance.

5. The voltage and reactive power control method for a microgrid according to claim 1, characterized in that, The lightweight decision model is based on multi-agent deterministic policy gradients, combined with a local-first experience replay mechanism. It uses a pre-defined differentiable projection module to constrain control actions, and is trained through asynchronous gradient aggregation, structured pruning, and low-bit-width quantization. Specifically: Based on the distribution network topology and the equipment parameters of distributed energy nodes, each intelligent agent is configured for each distributed energy node, and the policy network and value network of each intelligent agent are initialized. This enables each intelligent agent to generate original control actions based on local real-time operating data, and to map the original control actions to a feasible range that meets the constraints of voltage stability margin and reactive power balance through a preset differentiable projection module, thereby obtaining constrained feasible actions. The action is applied to the corresponding distributed energy node device. After a control interaction is completed, the latest local observation data and reward value after the interaction are obtained. Combined with the initial local observation data before the control interaction and the action, an experience data set is constructed. After initializing the experience priority of the experience data in the experience data group according to the preset rules, it is stored in the local priority experience playback buffer. Sampling with replacement is performed on the experience data from the local priority experience replay buffer according to the experience priority, and the policy gradient and value gradient are calculated based on the sampled experience data. Based on the policy gradient and value gradient, and combined with asynchronous gradient aggregation, structured pruning, and low-bit-width quantization, the lightweight decision model is trained.

6. The voltage and reactive power control method for a microgrid according to claim 5, characterized in that, The lightweight decision model is trained based on the policy gradient and value gradient, combined with asynchronous gradient aggregation, structured pruning, and low-bit-width quantization. Specifically: When the preset synchronization condition is met, based on the policy gradient and value gradient, the preset historical gradient momentum term is updated using the moving average method, and the parameters of the policy network and value network are adjusted and updated using the preset learning rate. The asynchronous gradients are aggregated to obtain the first parameter. Channel-level structured pruning is performed on the first parameter, and the norms of the output channel weights of each network layer of the policy network and the value network are calculated as each importance score. Sort all output channels by importance scores from low to high, select the output channel with the lowest importance score according to a preset ratio, and remove the input channels corresponding to the output channel with the lowest importance score. Perform structured pruning on the policy network and value network. Based on the symmetric quantization method, the scaling factor is calculated by combining the maximum absolute value of the weights of the network parameters of the pruned policy network and value network. The pruned network parameters are quantized to the range of signed integers using the scaling factor to generate lightweight model parameters. The lightweight model is trained based on the parameters of the lightweight model.

7. A voltage and reactive power control system for a microgrid, characterized in that, include: The acquisition module is used to acquire the distribution network topology, distributed energy node equipment parameters, and real-time operating data of each node. The configuration module is used to configure each intelligent agent and each constraint boundary for each distributed energy node based on the topology and distributed energy node device parameters. The processing module is used to input the real-time running data of each node into a preset lightweight decision model based on each intelligent agent and each constraint boundary, so that the lightweight decision model outputs a continuous reactive power control command that satisfies the voltage stability margin and reactive power balance constraints. The lightweight decision model is based on multi-agent deterministic policy gradient, combined with a local priority experience replay mechanism, and is obtained by constraining and mapping control actions through a preset differentiable projection module, and then training through asynchronous gradient aggregation, structured pruning and low bit-width quantization. The control module is used to perform closed-loop control of the distributed energy node equipment in the distribution network according to the continuous reactive power control command.

8. The voltage and reactive power control system for a microgrid according to claim 7, characterized in that, The processing module includes: The input unit is used to combine the real-time running data of each intelligent agent with the corresponding node to obtain each first intelligent agent, and input each first intelligent agent into the built-in policy network of the lightweight decision model to generate the original control action. The projection unit is used to map the original control actions through the differentiable projection module integrated in the lightweight decision model, combined with the constraint boundaries corresponding to each first agent, to obtain feasible control actions that meet the constraints of voltage stability margin and reactive power balance. The output unit is used to summarize the feasible control actions of each first agent and output continuous reactive power control commands.

9. The voltage and reactive power control system for a microgrid according to claim 7, characterized in that, The processing module includes: The configuration unit is used to configure each agent for each distributed energy node based on the distribution network topology and the device parameters of the distributed energy nodes, and to initialize the policy network and value network of each agent. The mapping unit is used to enable each intelligent agent to generate original control actions based on local real-time running data, and to map the original control actions to a feasible range that meets the constraints of voltage stability margin and reactive power balance through a preset differentiable projection module, so as to obtain constrained feasible actions. The construction unit is used to apply the actionable actions to the corresponding distributed energy node device. After completing a control interaction, the latest local observation data and reward value after the interaction are obtained. Combined with the initial local observation data before the execution of a control interaction and the actionable actions, an experience data group is constructed. The storage unit is used to initialize the experience priority of the experience data in the experience data group according to preset rules and then store it into the local priority experience playback buffer. The sampling unit is used to sample the experience data with replacement from the local priority experience replay buffer according to the experience priority, and calculate the policy gradient and value gradient based on the sampled experience data. The training unit is used to train the lightweight decision model based on the policy gradient and value gradient, combined with asynchronous gradient aggregation, structured pruning and low bit-width quantization.

10. The voltage and reactive power control system for a microgrid according to claim 9, characterized in that, The training unit includes: The update subunit is used to update the preset historical gradient momentum term based on the policy gradient and value gradient when the preset synchronization condition is met, combined with the moving average method, and to adjust and update the parameters of the policy network and value network in combination with the preset learning rate, and to aggregate the asynchronous gradient to obtain the first parameter. The pruning subunit is used to perform channel-level structured pruning on the first parameter and calculate the norm of each channel weight of each network layer of the policy network and the value network as each importance score. The sorting subunit is used to sort the importance scores of all output channels from low to high, select the output channel with the lowest importance score according to a preset ratio, and remove the input channels corresponding to the connection relationship of the output channel with the lowest importance score, thereby performing structured pruning on the policy network and the value network. The computational subunit is used to calculate the scaling factor based on the maximum absolute value of the weights of the network parameters of the pruned policy network and value network using the symmetric quantization method. A subunit is generated to quantize the pruned network parameters to a signed integer representation range using the scaling factor, thereby generating lightweight model parameters. The training subunit is used to train the lightweight model based on the parameters of the lightweight model.

Images