Power distribution network dynamic reactive power optimization control system based on reinforcement learning

Abstract

The application belongs to the field of power system automation and intelligent control, and specifically discloses a dynamic reactive power optimization control system of a distribution network based on reinforcement learning. The system comprises a data acquisition device, a physical information neural network module, a graph reinforcement learning decision module, an action safety verification unit and a reactive power regulation execution unit. The physical consistency of the control action is quickly evaluated through the neural network embedded with the electric power physical law, the multi-agent collaborative decision is realized in combination with the graph attention mechanism, the safety verification unit is used to shield the out-of-limit action, and finally the capacitor bank, the on-load voltage regulating transformer and the distributed power source are driven to perform dynamic reactive power regulation. Through the above technical scheme, the learning efficiency and the control robustness are improved under the premise of ensuring the safety of the system, and the weak perception and strong fluctuation operation environment are adapted.

Description

Technical Field

[0001] This invention belongs to the field of power system automation and intelligent control, specifically relating to a dynamic reactive power optimization control system for distribution networks based on reinforcement learning. Background Technology

[0002] With the continuous evolution of modern power distribution network architecture, the large-scale integration of distributed renewable energy has led to a high degree of randomness and volatility in the operation of the power grid. Dynamic reactive power optimization of the distribution network, as a core task for ensuring voltage stability, reducing line losses, and improving power supply quality, has profound strategic significance for the safe and efficient operation of the power system. Typically, by coordinating and controlling various reactive power compensation devices and regulation methods, it is possible to smooth system voltage deviations and optimize power flow distribution.

[0003] Utilizing artificial intelligence, especially deep reinforcement learning, for real-time decision-making and control of power distribution networks has become a core technological direction for solving high-dimensional nonlinear optimization problems. These methods, by constructing Markov decision processes, aim to enable intelligent agents to learn dynamic adjustment strategies to cope with complex operating conditions through interaction with massive historical data and simulation environments, achieving millisecond-level precise scheduling of reactive power compensation components such as capacitor banks, on-load tap-changing transformers, and distributed power inverters.

[0004] Traditional data-driven reinforcement learning models exhibit strong high-dimensional exploration blindness in the early stages of training. The control commands they generate are prone to violating Kirchhoff's laws or exceeding the rated safety limits of physical equipment, resulting in extremely slow training convergence and a serious risk of physical equipment damage. Simultaneously, existing algorithms lack the ability to deeply mine training samples. Due to the lack of effective fusion of the correlation between the physical topology of the distribution network, the exploration space becomes redundant in large-scale node systems, making efficient sample transformation difficult. In weak sensing scenarios with sparse deployment of measurement units, existing control frameworks lack the ability to infer the missing states of key nodes, resulting in poor control robustness in heterogeneous and incomplete data environments, making it difficult to support the stable operation of highly dynamic distribution networks.

[0005] Therefore, a dynamic reactive power optimization control system for power distribution networks based on reinforcement learning is desired. Summary of the Invention

[0006] The purpose of this invention is to provide a dynamic reactive power optimization control system for power distribution networks based on reinforcement learning, which can solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: The dynamic reactive power optimization control system for distribution networks based on reinforcement learning includes a data acquisition device, a physical information neural network module, a graph reinforcement learning decision-making module, an action safety verification unit, and a reactive power control execution unit, wherein: The data acquisition device is configured to acquire the voltage, current, power and topological connection information of each node in the distribution network in real time, and transmit the acquired data to the physical information neural network module and the graph reinforcement learning decision module. The physical information neural network module is based on the basic physical laws of power systems, especially Kirchhoff's current law and voltage law, and constructs a neural network model with embedded differential equation constraints. This model is used to quickly assess the physical consistency of the system state caused by any candidate control action without relying on complete power flow calculation. The graph reinforcement learning decision module adopts a multi-agent architecture and combines graph attention mechanism to model the topology of the distribution network. Each agent generates a preliminary reactive power regulation strategy based on local observation information and the state interaction of neighboring nodes, and outputs a candidate action sequence. The action safety verification unit receives the candidate actions output by the graph reinforcement learning decision module and calls the physical information neural network module to make a physical feasibility judgment on the system response corresponding to the action. If the action causes the system state to violate the preset safety constraints or physical laws, a negative reward signal is applied to the action or it is directly blocked. The reactive power control execution unit sends reactive power adjustment signals to capacitor banks, on-load tap-changing transformers, and distributed power inverters in the distribution network according to the final action command after safety verification, so as to realize dynamic optimization of system voltage and power flow distribution.

[0008] Preferably, the physical information neural network module embeds the steady-state power flow equation of the distribution network as a soft constraint into its loss function, so that the network automatically learns the state mapping relationship that satisfies the physical laws during the training process. In the case of missing measurements of some input nodes, it can infer the reasonable state range of the unmeasured nodes based on the electrical quantities of adjacent nodes.

[0009] Furthermore, each agent in the graph reinforcement learning decision module corresponds to a controllable reactive power resource node in the distribution network. Its observation space includes the local voltage amplitude, injected power, and connection weights with neighboring nodes. Its action space is the output setpoint of the adjustable reactive power equipment of the node. Agents dynamically allocate the importance weights of neighbor information through a graph attention mechanism to achieve collaborative perception of the global state.

[0010] Furthermore, when determining whether a candidate action exceeds the limit, the action safety verification unit relies on the voltage upper and lower limit thresholds, equipment capacity limits, and line thermal stability boundaries preset by the operating procedure. All limit exceedance judgments are completed through the system state estimation value output by the physical information neural network module, avoiding the frequent calls to the computationally complex power flow calculation program in traditional methods.

[0011] Preferably, in weakly perceptual areas where some measurement units are missing, the data acquisition device can still complete the data through the state inference results provided by the physical information neural network module, ensuring the strategy generation capability of the graph reinforcement learning decision module in a non-fully observable environment.

[0012] Furthermore, the reactive power control execution unit is equipped with a feedback correction mechanism, which can transmit the actual system response data after execution back to the graph reinforcement learning decision module for online updating of the agent policy network parameters, thereby achieving continuous adaptive optimization of the control strategy.

[0013] Furthermore, there is a bidirectional coupling relationship between the physical information neural network module and the graph reinforcement learning decision module: on the one hand, the physical information neural network provides guidance signals for safe exploration for reinforcement learning; on the other hand, the high-quality state-action pairs generated by reinforcement learning during policy optimization can also be used to enhance the physical information neural network's ability to generalize and fit physical laws under complex working conditions.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention fundamentally constrains the exploration space of the agent by deeply embedding a physical information neural network into a graph reinforcement learning framework, reducing training risks and convergence time caused by ineffective or excessive trial and error, and avoiding security threats to physical devices; it utilizes physical laws to impose prior constraints on the action space, improving learning efficiency under limited samples, and enabling the system to have stronger sample economy in real operating environments; in weak perception scenarios with incomplete measurement information, the physical information neural network can infer the state of missing nodes based on the power grid topology and physical correlation, enhancing the robustness and adaptability of the entire control system; 2. This invention uses a graph attention mechanism to explicitly model the distribution network structure, enabling efficient collaborative decision-making among multiple agents and overcoming the dimensionality curse problem faced by traditional centralized methods in large-scale systems. The overall architecture integrates the physical mechanisms of power systems, applied mathematical solution theory, and deep reinforcement learning decision logic, breaking down disciplinary barriers and providing a new technical path that combines security, efficiency, and intelligence for dynamic reactive power optimization of distribution networks under high-proportion renewable energy access. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the overall technical solution architecture of the present invention; Figure 2 This is a schematic diagram of the core principle framework of the bidirectional coupling between physical information neural network and graph reinforcement learning in this invention; Figure 3 This is a flowchart illustrating the logical process of the graph reinforcement learning decision module in this invention, which performs multi-agent collaborative perception and preliminary strategy generation based on graph attention mechanism. Figure 4 This is a flowchart illustrating the logical process of the motion safety verification unit in this invention calling the physical information neural network to evaluate system response and determine physical feasibility. Figure 5 This is a schematic diagram of the core interaction relationships and data flow between the data acquisition device, the reactive power control execution unit, and each core module in this invention. Detailed Implementation

[0016] Example 1: Please refer to the appendix Figure 1 To be continued Figure 5 To make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments.

[0017] The dynamic reactive power optimization control system for power distribution networks based on reinforcement learning includes a data acquisition device, a physical information neural network module, a graph reinforcement learning decision module, an action safety verification unit, and a reactive power control execution unit. The data acquisition device is configured at various key physical nodes of the distribution network to acquire in real time the voltage amplitude, current phase, active power, reactive power, and branch topology information of each node in the distribution network. The data acquisition device includes multiple distributed synchronous vector measurement units and miniature intelligent sensors. These units establish bidirectional data links with the physical information neural network module and graph reinforcement learning decision module through a high-speed fiber optic backbone network or industrial wireless communication network. During operation, the data acquisition device synchronously samples electrical quantities at a preset sampling frequency, configured between 50 and 200 times per second, to ensure that the transient response characteristics of the distribution network under distributed energy fluctuations can be captured. In addition, the data acquisition device also integrates a data preprocessing unit to perform noise reduction, filtering, and outlier removal on the acquired raw electrical signals to ensure that the data entering the subsequent calculation module has high fidelity.

[0018] The physical information neural network module, connected to the data acquisition device, is configured to construct a deep neural network model with embedded differential equation constraints based on the fundamental physical mechanisms of the power system. The core of this module lies in transforming Kirchhoff's current law and voltage law into intrinsic constraints of the neural network, achieving rapid extrapolation and prediction of the system state without relying on traditional, computationally complex Newton-Raphson power flow calculation programs. The physical information neural network module includes a physical law embedding unit, a differential equation solving unit, and a state inference unit. The physical law embedding unit calculates the steady-state power flow of the distribution network... The flow equation is defined in the network structure as a loss function term. Specifically, it transforms the balance between node injection power and branch flow power into an algebraic constraint residual, requiring this residual to tend to a minimum during network training. The differential equation solving unit uses automatic differentiation technology to calculate the partial derivatives of the network output state variables with respect to the input control variables, constructing a state mapping space that satisfies physical continuity. The state inference unit is used to deduce the reasonable voltage range and power distribution of unmonitored nodes based on the physical correlation and topology of adjacent nodes in scenarios where there are partial missing or noise interference in the input data.

[0019] The graph reinforcement learning decision module employs a deep reinforcement learning algorithm based on a multi-agent architecture and combines it with a graph attention mechanism to explicitly model the complex topology of the distribution network. This module includes a topology representation unit, an agent interaction unit, a policy generation unit, and a reward evaluation unit. The topology representation unit abstracts each transformer node, capacitor node, and distributed power source access point in the distribution network as vertices in a graph structure, and the transmission and distribution lines as edges in the graph, using graph convolution or graph attention operations to extract spatial correlation features between nodes. The agent interaction unit assigns an independent agent to each controllable reactive power compensation device, and the agents share the state information of neighboring nodes through an interaction mechanism. The policy generation unit is configured to output a reactive power adjustment action sequence for specific devices based on the current system observation state, using a multi-layer fully connected neural network or a recurrent neural network. This action sequence includes the number of capacitor switching groups, the adjustment position of transformer taps, and the reactive power output setpoint of the inverter. The reward evaluation unit calculates the instantaneous reward score for each agent based on the reduction in active power loss, the reduction in voltage deviation, and the smoothness of the control actions.

[0020] The action safety verification unit, located between the graph reinforcement learning decision module and the reactive power control execution unit, is configured to perform a preliminary safety review of the candidate actions output by the graph reinforcement learning decision module. This unit rapidly simulates the system state changes that may be triggered after the candidate action is executed by calling the physical information neural network module in real time. If the simulation results show that the action will cause the voltage of some nodes to exceed a preset safety threshold, or cause the line current to exceed its thermal stability limit, the action safety verification unit will automatically trigger a shielding mechanism to prevent the action from being issued and will feed back a large negative penalty signal to the graph reinforcement learning decision module to guide the agent to avoid such unsafe search areas in subsequent iterations. The judgment logic of the safety verification is based on a strategy combining soft and hard constraints, that is, applying gradual reward decay for minor violations of limits, while executing a veto for violations of physical laws.

[0021] The reactive power regulation execution unit receives the final control command filtered by the action safety verification unit and converts it into a corresponding physical execution signal, which is then sent to the primary equipment in the distribution network. The reactive power regulation execution unit includes an actuator driver, an instruction verifier, and a feedback monitoring unit. The actuator driver is used to drive the electromagnetic contactor of the capacitor bank, the on-load tap changer of the on-load tap-changing transformer, and the power control chip of the distributed power grid-connected inverter. The instruction verifier is used to confirm the validity and timing consistency of the instruction before execution to prevent malfunctions caused by communication delays. The feedback monitoring unit monitors the system response after execution in real time and sends the actual voltage improvement data and line loss change data back to the graph reinforcement learning decision module for online optimization of the agent's strategy network parameters, forming a complete closed-loop adaptive control system.

[0022] During the construction of the physical information neural network module, its loss function is designed to include a combination of data-driven terms and physical constraint terms. The data-driven terms are defined as the sum of the mean square errors between the node voltages predicted by the neural network and the measured node voltages. The physical constraint terms are defined as the sum of the absolute values ​​of the residuals of the power balance equations constructed based on Kirchhoff's laws. During training, by adjusting the weight ratio of the physical constraint terms in the total loss function, the model can still strictly follow the electrical and physical characteristics of the power distribution network even with a small sample size.

[0023] The graph attention mechanism in the graph reinforcement learning decision module is configured as a dynamic weight allocation process. Specifically, for any target node in the distribution network, the mechanism automatically learns the contribution of each neighboring node to the voltage fluctuation of the target node by calculating the dot product similarity between the feature vector of the target node and the feature vectors of its neighbors. Subsequently, the normalized attention weights are used to weighted aggregate the neighborhood information to generate a node feature representation containing global spatial constraints. This approach ensures that when the distribution network topology undergoes structural changes (such as line switch operation), the control system can quickly adapt to the new operating environment by updating the adjacency matrix.

[0024] Furthermore, when the action safety verification unit performs the physical feasibility judgment, the physical laws it relies on include at least the following: First, the algebraic sum of voltage drops in all closed loops within the system must be equal to zero; second, the algebraic sum of currents flowing into any node must be equal to zero; third, the output power of all power generation equipment and reactive power compensation equipment must not exceed their rated capacity limits; fourth, the per-unit current value of all feeder segments must not exceed their designed thermal stability limit per-unit value.

[0025] Furthermore, the reactive power regulation execution unit is configured with smoothing logic based on time steps. Since capacitor banks and transformer taps are discrete regulation devices with limited number of operations, the smoothing logic is configured to: count the cumulative number of operations of a specific device within a given preset time period; if the number reaches a preset life protection threshold, the regulation function of the device is temporarily locked, and the reactive power regulation demand is transferred to the inverter unit with continuous regulation capability, thereby extending the service life of the physical equipment and reducing maintenance costs.

[0026] There is a deep bidirectional coupling and interaction mechanism between the physical information neural network module and the graph reinforcement learning decision module. On the one hand, the physical information neural network provides gradient guidance for the reinforcement learning agent by providing accurate system sensitivity analysis, enabling it to perceive the nonlinear mapping relationship between reactive power regulation actions and voltage improvement effects more quickly. On the other hand, the large number of exploratory samples generated during the training process of reinforcement learning cover the operating states of the distribution network under various extreme conditions. These high-quality state-action pairs are fed back to the physical information neural network to further correct the parameters of the physical constraint terms and improve the neural network's ability to generalize and model complex nonlinear power systems.

[0027] At the hardware implementation level, the system can be deployed in the central server cluster of the power distribution network dispatch center, or distributed in the edge computing platforms of each substation. The central server cluster adopts a heterogeneous parallel architecture of multi-core central processing units and large-scale graphics processors to support the ultra-large-scale parallel computing requirements of physical information neural networks. The edge computing platform adopts a low-power embedded processor system, which is responsible for executing local graph reinforcement learning inference logic and real-time action security interception.

[0028] Example 2: As a further supplement or alternative to Example 1, this example describes a dynamic reactive power optimization control system for distribution networks based on an edge-cloud collaborative architecture, which aims to solve the problem of balancing communication latency and computing pressure in large-scale distribution networks.

[0029] Under this architecture, the system includes a cloud-based training platform, multiple edge processing nodes, and on-site execution terminals; The cloud-based training platform, deployed on the power company's private cloud server cluster, is primarily responsible for the offline pre-training of the physical information neural network module and the deep policy iteration of the graph reinforcement learning decision module. The cloud-based training platform possesses powerful computing resources, capable of processing massive amounts of historical operational data and simulating various extreme fault scenarios using complex physical models. In the cloud, the physical information neural network module learns the full topology model of the distribution network to construct a high-precision digital twin system. This system not only includes basic Kirchhoff's laws constraints but also incorporates the frequency-dependent characteristics of transmission lines and the nonlinear transformer excitation characteristics.

[0030] The edge processing nodes are deployed in substations or branch boxes at all levels of the power distribution network, and are responsible for performing real-time state perception and local reactive power optimization decisions. Each edge processing node integrates a lightweight physical information neural network model and a local graph attention inference unit. The edge processing nodes are directly connected to the data acquisition device and can acquire the electrical parameters of the local area and its neighboring branches with extremely low latency. When a risk of exceeding the local voltage limit is detected, the edge processing nodes can generate instant reactive power adjustment commands using the pre-stored strategy network without uploading to the cloud.

[0031] The field execution terminal communicates with the edge processing node through a bus structure or wireless mesh network, and is responsible for converting control commands into specific actions of driving motors or power electronic devices. The field execution terminal also has a health status self-diagnosis function, which can monitor the operating temperature, insulation impedance and other physical indicators of the compensation equipment in real time, and feed these health indicators back to the action safety verification unit of the edge processing node as constraints.

[0032] The edge processing nodes and the cloud training platform adopt an asymmetric synchronization mechanism. The cloud training platform pushes updated neural network weight parameters to each edge node periodically (e.g., hourly or daily) according to the overall network operation trend. The edge nodes, in turn, desensitize new operating condition data or strategy deviation data encountered in real-time control and asynchronously upload them to the cloud to improve the global model in the cloud. This architecture effectively reduces the dependence on backbone communication bandwidth and also ensures the system's local autonomous control capability in extreme situations such as communication interruption.

[0033] Within the internal logic of the edge processing node, the action safety verification unit is given higher priority. This unit employs a safety assessment algorithm based on interval prediction, utilizing the probability interval of the output voltage distribution from a physical information neural network, rather than a single point estimate. Only when all candidate actions meet the physical safety constraints at a preset confidence level are the actions allowed to be executed. This probability-based verification mechanism greatly improves the system's robustness in dealing with the randomness of distributed power output.

[0034] Furthermore, the graph reinforcement learning decision module adopts a hierarchical reinforcement learning architecture at the edge; the bottom-level agent is responsible for controlling the micro-adjustment of individual devices, such as the power factor control of the inverter; the top-level agent is responsible for the coordination of multiple devices in the substation area, coordinating the action sequence of capacitor banks and voltage regulating transformers through graph attention mechanism, thereby avoiding adjustment conflicts and oscillations between different devices.

[0035] Furthermore, in Embodiment 2, the data acquisition device adds a feature extraction function based on edge intelligence. This function uses a simple convolution kernel to perform sliding window processing on the original waveform, automatically identifies and extracts feature vectors such as voltage sag and harmonic exceedance, and sends only the feature vectors to the physical information neural network module, thus compressing the amount of data transmission while ensuring information integrity.

[0036] Example 3: This example describes a variant of the system architecture for a weakly sensing distribution network environment with a high proportion of renewable energy access and highly sparse measurement points; In this scenario, the physical information neural network module is configured as a physical inference engine with "data completion" function. Since a large number of end nodes in the weak sensing distribution network lack real-time monitoring equipment, the physical inference engine utilizes the topological correlation of the distribution network, takes the voltage and current of the known monitoring nodes as boundary conditions, and reconstructs the voltage distribution field of the entire network by solving the partial differential equations embedded in the network. This reconstruction process strictly follows the physical reduction of the laws of charge conservation and energy conservation.

[0037] In this embodiment, the graph reinforcement learning decision module introduces a partially observable Markov decision process model. Since the agent cannot obtain the complete state of the entire network, the graph reinforcement learning decision module integrates historical observation sequences through recurrent neural network units to construct a hidden state space to represent the operating trend of the system. The graph attention mechanism is used to establish causal relationships between nodes in the hidden state space, enabling the agent to deduce a reactive power compensation strategy with global optimal potential based on local and fragmented information.

[0038] In Embodiment 3, the action safety verification unit has particularly strengthened the modeling of "physical uncertainty". When the physical information neural network module shows high uncertainty in the evaluation result of a candidate action (for example, due to the lack of key observation points leading to excessive inference variance), the safety verification unit will implement a conservative strategy, that is, shrink the action space to the known safe and stable domain until more data input is obtained to reduce the evaluation risk.

[0039] In addition, the reactive power control execution unit in this embodiment adds scheduling logic for the reactive power support capability of the distributed energy storage system; the system regards the remaining capacity of the energy storage inverter as a flexible reactive power regulation resource. When the regulation margin of the capacitor bank is insufficient or the response speed cannot meet the instantaneous fluctuation requirements, the reactive power output of the energy storage inverter is precisely controlled through graph reinforcement learning strategy, thereby improving the distribution network's ability to resist dynamic disturbances without increasing additional hardware investment.

[0040] The bidirectional coupling between the physical information neural network module and the graph reinforcement learning decision module manifests as a "generative adversarial" evolutionary process. The graph reinforcement learning, acting as a generator, continuously attempts to generate control actions that can reduce line loss. Meanwhile, the physical information neural network, acting as a discriminator, performs rigorous legality judgment and state deduction on these actions based on physical laws. The two evolve together in mutual competition, resulting in a final control scheme that possesses both powerful optimization performance and unquestionable physical security.

[0041] Example 4: This example details the engineering implementation of the reinforcement learning-based dynamic reactive power optimization control system for power distribution networks in terms of hardware integration and communication protocols to ensure the system's practicality and reliability.

[0042] The system is deployed in a highly integrated industrial-grade control cabinet, which is equipped with a high-performance embedded computing unit, multiple serial communication interfaces, an industrial Ethernet switch, and an uninterruptible power supply module. The embedded computing unit adopts a heterogeneous multi-core architecture, which includes four high-performance general-purpose processing cores for running the operating system and conventional logic, and an integrated tensor processing unit for accelerating the inference calculation of physical information neural networks and graph reinforcement learning models. The memory system of the computing unit adopts synchronous dynamic random access memory with error correction code function to prevent calculation errors caused by bit flips in power distribution environments with high electromagnetic interference.

[0043] The communication interface supports multiple power industry standard protocols, including but not limited to the IEC61850 modeling standard, the DL / T645 energy meter communication protocol, and the Modbus-RTU industrial protocol. The system communicates bidirectionally with intelligent electronic devices distributed throughout the distribution network through these interfaces. To ensure the security of data transmission, all control commands are encrypted before being sent and are accompanied by timestamp verification information to prevent replay attacks and malicious tampering.

[0044] At the software architecture level, the system runs on a microkernel operating system optimized for real-time performance. The operating system divides data acquisition, physical information prediction, decision generation, and security verification into different privileged tasks, and ensures that the security verification task can be completed within millisecond time slices through a strict priority scheduling algorithm, thus meeting the real-time requirements of dynamic reactive power regulation of the distribution network.

[0045] The training process of the physical information neural network module adopts a hybrid precision training technique, which uses half-precision floating-point numbers to perform large-scale matrix multiplication and addition operations while keeping the key weight parameters as single-precision floating-point numbers. This method increases the inference speed of the model by more than two times without losing almost the model's prediction accuracy, enabling the system to respond in real time to instantaneous voltage drops caused by sudden photovoltaic shading in the distribution network.

[0046] Furthermore, the graph reinforcement learning decision module integrates an experience replay buffer, which is configured to store samples hierarchically according to their importance. For failed experiences that include attempts to exceed voltage limits and were successfully intercepted by the action safety verification unit, the system assigns them higher sampling weights. By reinforcing the learning of failed cases, the agent can establish a strong sense of respect for physical boundaries with extremely high efficiency, resulting in higher safety in actual operation.

[0047] Furthermore, the action safety verification unit internally stores a complete set of power distribution network physical model parameter library, including line resistance, reactance, susceptance parameters, and detailed mathematical descriptions of various compensation devices; this library can be dynamically corrected according to factors such as ambient temperature and years of operation, ensuring that the parameters used by the physical information neural network to perform state inference are highly consistent with the actual equipment characteristics.

[0048] In addition, the system is also equipped with a web-based human-computer interaction interface, through which dispatchers can view the full network voltage cloud map derived by the physical information neural network, the decision trajectory of the graph reinforcement learning agent, and the operation statistics of each compensation device in real time. When the system performs a critical reactive power adjustment action or successfully intercepts an unsafe action, the interaction interface will generate corresponding event records and audible and visual alarm prompts.

[0049] In summary, this invention fundamentally constrains the exploration space of agents by deeply embedding a physical information neural network into a graph reinforcement learning framework, significantly reducing training risks and convergence time caused by ineffective or excessive trial and error, and avoiding security threats to physical equipment. By utilizing physical laws to impose prior constraints on the action space, it improves learning efficiency under limited samples, enabling the system to have stronger sample economy in real-world operating environments. In weak perception scenarios with incomplete measurement information, the physical information neural network can infer the state of missing nodes based on the power grid topology and physical correlations, enhancing the robustness and adaptability of the entire control system. Through explicit modeling of the distribution network structure using a graph attention mechanism, it achieves efficient collaborative decision-making among multiple agents, overcoming the dimensionality curse problem faced by traditional centralized methods in large-scale systems. The overall architecture integrates the physical mechanisms of power systems, applied mathematical solution theory, and deep reinforcement learning decision-making logic, breaking down disciplinary barriers and providing a novel technical path for dynamic reactive power optimization of distribution networks with high-proportion renewable energy access, combining safety, efficiency, and intelligence.

[0050] In actual deployment, the logical and physical connections between the modules of the system of this invention are designed with redundancy. For example, the data acquisition device is equipped with a backup wireless communication channel. When the wired fiber optic link fails, the system can automatically switch to wireless mode to maintain basic state awareness. When the reactive power control execution unit receives an abnormal adjustment command (such as frequent repetitive actions), it will automatically enter a locked state and send a manual intervention request to the management center to ensure the absolute safety of the power grid operation.

[0051] In terms of algorithm evolution, the physical information neural network used in this invention is not limited to handling steady-state power flow constraints. Its internal differential equation descriptors can be further expanded to include transient stability differential algebraic equations, enabling the system to have the ability to cope with dynamic reactive power support and voltage recovery control after a short-circuit fault in the distribution network, demonstrating extremely high technical scalability and foresight.

[0052] Finally, it should be noted that each module, unit, and specific logical step in this invention can be flexibly tailored and reorganized according to the actual scale of the distribution network and the computing power configuration. For example, in some smaller microgrid systems, the physical information neural network module and the graph reinforcement learning decision module can be merged and run in a single high-performance controller. In large-scale active distribution networks across regions, a distributed edge computing approach is preferred.

[0053] Those skilled in the art should understand that the above-described embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

[0054] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A dynamic reactive power optimization control system for power distribution networks based on reinforcement learning, characterized in that, It includes a data acquisition device, a physical information neural network module, a graph reinforcement learning decision module, an action safety verification unit, and a reactive power control execution unit; The data acquisition device is configured to acquire in real time the voltage amplitude, current phase, power information and topological connection information of each node in the distribution network, and transmit the acquired data to the physical information neural network module and the graph reinforcement learning decision module. The physical information neural network module is connected to the data acquisition device and is configured to construct a neural network model with embedded differential equation constraints based on the physical laws of the power system. Under the condition of not relying on the complete power flow calculation, it performs a physical consistency assessment of the system state caused by any candidate control action. The graph reinforcement learning decision module is connected to the data acquisition device and is configured to model the topology of the power distribution network using a multi-agent architecture combined with a graph attention mechanism. Each agent generates a preliminary reactive power regulation strategy based on local observation information and the state interaction of neighboring nodes, and outputs a candidate action sequence. The action safety verification unit is connected to the graph reinforcement learning decision module and the physical information neural network module respectively. It is configured to receive the candidate action sequence and call the physical information neural network module to perform a physical feasibility judgment on the system response corresponding to the candidate action sequence, so as to output the final action command after safety verification. The reactive power regulation execution unit is connected to the action safety verification unit and is configured to send adjustment signals to the reactive power compensation equipment in the distribution network according to the final action command, so as to realize dynamic optimization of system voltage and power flow distribution.

2. The dynamic reactive power optimization control system for power distribution networks based on reinforcement learning according to claim 1, characterized in that, The data acquisition device includes multiple distributed synchronous vector measurement units, smart sensors, and a data preprocessing unit. The synchronous vector measurement unit and the smart sensor establish a bidirectional data link with the physical information neural network module through an optical fiber network or a wireless communication network. The synchronous vector measurement unit is configured to synchronously sample electrical quantities at a preset sampling frequency, which is set to between 50 and 200 times per second. The data preprocessing unit is configured to perform noise reduction, filtering, and outlier removal on the acquired raw electrical signals. In areas of the power distribution network where measurements are missing and the sensing is weak, the data acquisition device is configured to use the state inference results provided by the physical information neural network module to complete the data.

3. The dynamic reactive power optimization control system for power distribution networks based on reinforcement learning according to claim 1, characterized in that, The physical information neural network module includes a physical law embedding unit, a differential equation solving unit, and a state inference unit; The physical law embedding unit is configured to define the steady-state power flow equations of the distribution network in the form of loss function terms in the network structure. The steady-state power flow equations include Kirchhoff's current law and Kirchhoff's voltage law. The differential equation solving unit is configured to use automatic differentiation technology to calculate the partial derivatives of the network output state variables with respect to the input control variables, and to construct a state mapping space that satisfies physical continuity. The state inference unit is configured to deduce the voltage range and power distribution of unmonitored nodes based on the physical correlation and topology of adjacent nodes in scenarios where input data is missing or subject to noise interference.

4. The dynamic reactive power optimization control system for power distribution networks based on reinforcement learning according to claim 3, characterized in that, During the construction process, the total loss function of the physical information neural network module is constructed as a combined structure containing data-driven terms and physical constraint terms; The data-driven term is defined as the sum of the mean square errors between the node voltages predicted by the neural network and the measured node voltages; The physical constraint term is defined as the sum of the absolute values ​​of the residuals of the power balance equations constructed according to Kirchhoff's laws; During model training, by adjusting the weight ratio of the physical constraint term in the total loss function, the physical information neural network module can still follow the electrical and physical characteristics of the power distribution network even when the sample size is less than a preset sample threshold.

5. The dynamic reactive power optimization control system for power distribution networks based on reinforcement learning according to claim 1, characterized in that, The graph reinforcement learning decision module includes a topology representation unit, an agent interaction unit, a policy generation unit, and a reward evaluation unit. The topology representation unit is configured to abstract transformer nodes, capacitor nodes, and distributed power supply access points in the power distribution network as vertices in a graph structure, and power distribution lines as edges in the graph, and to extract spatial correlation features between nodes using graph attention operations. The intelligent agent interaction unit assigns an independent intelligent agent to each controllable reactive power compensation device, and the intelligent agents are configured to share the state information of neighboring nodes through an interaction mechanism. The strategy generation unit is configured to output a sequence of reactive power adjustment actions for reactive power compensation equipment through a deep neural network based on the current system observation state. The reward evaluation unit is configured to calculate the instantaneous reward score of each agent based on the reduction in active power loss during system operation, the degree of reduction in voltage offset, and the smoothness of control actions.

6. The dynamic reactive power optimization control system for power distribution networks based on reinforcement learning according to claim 5, characterized in that, The graph attention mechanism is configured as a dynamic weight allocation process; For a target node in the distribution network, the graph reinforcement learning decision module is configured to learn the contribution of each neighbor node to the voltage fluctuation of the target node by calculating the dot product similarity between the feature vector of the target node and the feature vectors of its neighbor nodes. The graph reinforcement learning decision module is also configured to use normalized attention weights to perform weighted aggregation of neighborhood information to generate node feature representations containing global spatial constraints. When the topology of the power distribution network undergoes a structural change, the graph reinforcement learning decision module adjusts the node feature representation by updating the adjacency matrix.

7. The dynamic reactive power optimization control system for distribution networks based on reinforcement learning according to claim 1, characterized in that, When performing a physical feasibility assessment, the action safety verification unit relies on the following physical laws: First, the algebraic sum of voltage drops in all closed loops within the system is equal to zero; second, the algebraic sum of currents flowing into any node is equal to zero; third, the output power of all power generation equipment and reactive power compensation equipment is not greater than their rated capacity limit; fourth, the per-unit current value of all feeder segments is not greater than their designed thermal stability limit per-unit value. The action safety verification unit is equipped with a shielding mechanism. If the evaluation result of the physical information neural network module shows that the candidate action causes the node voltage to exceed the preset safety threshold or the line current to exceed the thermal stability limit, the shielding mechanism is triggered to prevent the action from being issued, and a negative penalty signal is fed back to the graph reinforcement learning decision module.

8. The dynamic reactive power optimization control system for power distribution networks based on reinforcement learning according to claim 1, characterized in that, The reactive power control execution unit includes an actuator driver, an instruction verifier, and a feedback monitoring unit; The actuator driver is configured to drive the electromagnetic contactor of the capacitor bank, the on-load tap changer of the on-load tap-changing transformer, and the power control chip of the distributed power grid-connected inverter. The instruction verifier is configured to verify the validity and timing consistency of instructions before execution. The feedback monitoring unit is configured to monitor the system response after execution in real time and transmit the actual voltage improvement data and line loss change data back to the graph reinforcement learning decision module for online updating of the agent's policy network parameters. The control actions corresponding to the final action command include the number of capacitor banks switched on and off, the adjustment position of the transformer taps, and the reactive power output setting value of the inverter.

9. The dynamic reactive power optimization control system for distribution networks based on reinforcement learning according to claim 8, characterized in that, The reactive power control execution unit is configured with smoothing processing logic based on time step. The smoothing logic is configured as follows: Within a given preset time period, the cumulative number of actions of the equipment is counted. If the cumulative number of actions reaches the preset life protection threshold, the adjustment function of the equipment is temporarily locked, and the reactive power adjustment demand is transferred to the inverter unit with continuous adjustment capability to extend the service life of the physical equipment. The reactive power control execution unit is also equipped with a feedback correction mechanism, which can transmit the actual system response data after execution back to the graph reinforcement learning decision module for online updating of the agent policy network parameters.

10. The dynamic reactive power optimization control system for power distribution networks based on reinforcement learning according to claim 1, characterized in that, The physical information neural network module and the graph reinforcement learning decision module are configured to have a bidirectional coupled interaction relationship. The physical information neural network module is configured to provide gradient guidance to the graph reinforcement learning decision module by providing system sensitivity analysis, so as to perceive the nonlinear mapping relationship between reactive power regulation actions and voltage improvement effects; The graph reinforcement learning decision module is configured to feed back high-quality state-action pairs generated during the strategy optimization process to the physical information neural network module, thereby correcting the physical constraint parameters within the physical information neural network module and improving its generalization modeling capability for nonlinear power systems. The system is deployed in a server cluster in the power distribution network dispatch center, or distributed in an edge computing platform in a substation.

Images