A power distribution system voltage control method based on optimal historical selection and prediction
By combining droop control with an improved MADDPG algorithm and using an LSTM model for prediction and sensitivity correction, the problem of insufficient voltage regulation in traditional droop control in high-penetration renewable energy distribution systems is solved, achieving more accurate and reliable voltage regulation, reducing computational resource requirements, and improving the feasibility of the method.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- TIANJIN UNIV
- Filing Date
- 2026-02-26
- Publication Date
- 2026-06-05
Smart Images
Figure CN122159287A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of voltage control technology for power distribution systems, and in particular to a voltage control method for power distribution systems based on optimal historical selection and prediction. Background Technology
[0002] To achieve carbon neutrality, the penetration of renewable energy sources in the power system, such as solar PV and wind power, is steadily increasing, gradually replacing traditional synchronous generators. New energy generation systems are connected to the grid via power electronic converters; therefore, inverter-based distributed energy resources (DERs), such as solar PV systems and battery energy storage systems (BESS), can operate in all four quadrants. These solar PV and BESS DERs are managed using Q–V droop control according to grid codes. However, droop control alone is insufficient for effective voltage regulation in distribution systems with high penetration of renewable distributed energy sources due to their inherent instabilities.
[0003] To enhance system voltage regulation in modern distribution networks—especially those with high penetration rates of distributed energy resources (DERs) such as solar photovoltaic—a promising and effective strategy is to equip grid-connected renewable energy converters (such as photovoltaic inverters) with advanced control based on multi-agent deep reinforcement learning (MADRL) methods, particularly algorithms such as Multi-agent Deep Deterministic Policy Gradient (MADDPG) and Multi-agent proximal policy optimization (MAPPO). These learning-based, data-driven methods enable multiple inverters (considered as autonomous agents) to make distributed, coordinated decisions. By learning optimal reactive power injection / absorption strategies from interactions with the grid environment, MADRL can achieve superior voltage regulation performance compared to traditional droop control.
[0004] Studies show that MADRL is more effective than traditional methods in reducing voltage overruns, lowering network power losses, and providing better overall controllability—especially in situations with large renewable energy fluctuations, weak grid support, or partial observability. Traditional droop control schemes often face steady-state biases or coordination issues. Conceptually, MADRL shifts voltage control from traditional empirical rules (such as droop methods) to adaptive, cooperative control learned through trial and error in simulated environments. The centralized training and distributed execution paradigm allows agents to implicitly consider system-wide coupling and uncertainty without continuous real-time communication, thus making it scalable in large-scale distribution systems. However, a key limitation of pure multi-agent deep reinforcement learning methods lies in their inherent randomness and lack of guarantees, which can lead to inconsistent or unstable performance under different scenarios, operating conditions, or when encountering typical scenarios (such as fluctuations caused by extreme weather or unknown network topologies). Unlike deterministic droop control, which reliably produces the same output for the same local measurements, MADRL policies may occasionally produce suboptimal or unsafe actions during the exploration phase, early training, or simulation-to-real-world transfer.
[0005] To compensate for the reliability gap of MADRL in practical voltage regulation in distribution systems dominated by distributed energy resources, a key promising direction is Physics-informed MADRL. This is achieved by directly incorporating basic power flow equations, network topology, or physical priors into the learning process—typically through constraint embedding, reward shaping, or hybrid model components. This approach can produce more robust and stable performance: it enforces physically reasonable operations, more consistently reduces voltage violations, improves generalization to unseen conditions, and mitigates anomalous behavior compared to purely data-driven MADRL. However, its main drawbacks are increased algorithmic complexity (e.g., embedding GCNs, physics-based loss terms, or hybrid architectures). Longer training times and higher computational demands are required during both the training and inference phases, requiring more powerful hardware than standard multi-agent deep reinforcement learning (MADRL). Essentially, physical augmentation makes MADRL safer and more reliable in practical deployments, but at the cost of higher complexity and resource consumption—a trade-off that ongoing research aims to optimize. Therefore, a reliable voltage regulation method is needed that remains simple, eliminates the need for high-performance computing hardware, and still provides better performance than traditional droop control under the expected photovoltaic power generation and load conditions. Summary of the Invention
[0006] Therefore, one objective of this invention is to propose a voltage control method for power distribution systems based on optimal historical selection and prediction, in order to solve the problems mentioned in the background art and overcome the shortcomings of the prior art.
[0007] To achieve the above objectives, the present invention provides a voltage control method for a power distribution system based on optimal historical selection and prediction, comprising: Offline phase: Based on historical data, determine the optimal reactive power setpoint for each photovoltaic distributed energy node and build an optimal setpoint knowledge base; At this stage: Based on real-time data, predict the optimal reactive power setpoint for each photovoltaic distributed energy node in the current scenario; The predicted optimal reactive power setpoint is subjected to online iterative voltage correction based on sensitivity, and the corrected setpoint is sent to each photovoltaic inverter for execution to realize voltage regulation of the power distribution system.
[0008] As a preferred option, in the offline stage: based on historical photovoltaic power generation data and load data, corresponding reactive power setpoints are generated using rule-based droop control algorithms and improved MADDPG algorithms, respectively. The optimal reactive power setpoint with the minimum voltage deviation is selected for each photovoltaic distributed energy node, and an optimal setpoint knowledge base is constructed. Online phase: Acquire real-time photovoltaic power generation data and load data, match the closest historical scenario in the knowledge base by calculating the similarity with historical photovoltaic curves, and use a pre-trained long short-term memory neural network model to predict the optimal reactive power setpoint of each photovoltaic node in the current scenario.
[0009] Preferably, the improved MADDPG algorithm includes: Actor Network: Each photovoltaic node acts as an agent, with a policy function. For a neural network, the input is the local state. The output is reactive power operation. ; Commentator Network: Includes a centralized action-value function The input is the joint state of all agents. and combined effects The output is a Q-value, used to evaluate the state. The following joint actions were taken. The expected cumulative return; Target Network: Set up target networks for both actors and commentators. and And by updating the rules and Update.
[0010] Preferably, the actor network is updated via deterministic gradients, where the gradient is: ; in, The actor parameter represents the expected return relative to agent i. The gradient is given, the batch size is B, and the index j is the sample drawn from the playback buffer. The function represents the importance sampling or weighting factor of the j-th sample. It is the deterministic strategy (actor) of agent i, and its local observation Mapping to an action, the critique function It is a centralized action-value function with parameters. Used to evaluate the global state The combined effect of gradients The sensitivity of critics to agent i's actions is measured by an evaluation value of [value missing]. .
[0011] Preferably, the commentator network minimizes the time difference loss between the target Q-value and the predicted Q-value. Implement update: in, Let B represent the mean squared error loss function of the commenter network, B be the mini-batch size, and j be the sample index sampled from the empirical replay pool. Let j be the weight of the j-th sample. For parameters The centralized critic network represented in the state and joint actions Estimation of the action value function under the following conditions For the target Q value, It consists of immediate returns and discounted future returns, of which Indicates the total number of agents. The instantaneous reward obtained by the i-th agent in the j-th sample. As a discount factor, As a termination marker, This represents the global state at the next moment. For the target policy network, For the target commentator network.
[0012] Preferably, the improved MADDPG algorithm includes: Prioritize experience playback, and assign priorities based on the absolute value of the TD error of the experience samples; Adaptive noise that decays with the number of training rounds is used; Learning rate scheduling involves decaying the learning rate after a preset number of rounds.
[0013] Preferably, selecting the optimal reactive power setpoint with the minimum voltage deviation for each photovoltaic distributed energy node includes: For each reactive power setpoint, the corresponding total squared voltage deviation is calculated, and the total squared voltage deviation is based on the formula... Confirmed, among which The voltage of bus b at hour h. Reference voltage; Compare the total square voltage deviation of all reactive power setpoints and select the reactive power setpoint with the smallest deviation as the optimal setpoint.
[0014] Preferably, the input to the long short-term memory neural network model is normalized time-series data, including photovoltaic power generation data and load data, wherein the input vector is represented as... ,in This represents the total active power generation of the photovoltaic DER. t is the load scaling factor for time step t, corresponding to 144 time steps in 10 minutes; The output of the long short-term memory neural network model is the reactive power setpoint vector. .
[0015] Preferably, the long short-term memory neural network model includes two stacked LSTM layers, each layer containing 64 hidden units, with dropout regularization applied between layers, and the hidden states mapped to the output through a fully connected layer; The training process uses 200 days of synthetically generated data, with each day containing 144 time steps. The loss function is as follows: ; in, n is the number of photovoltaic distributed energy sources. The Adam optimizer is used with a learning rate of 0.003, and a total of 300 training cycles are performed. The input and output of the Long Short-Term Memory Neural Network Model are both normalized based on the mean and standard deviation calculated from the training set.
[0016] Preferably, the online iterative voltage correction based on sensitivity for the predicted optimal reactive power setpoint includes: For each time step, the voltage error of the photovoltaic access bus is calculated. If the absolute value of the error exceeds the dead zone threshold, the local voltage sensitivity is estimated by perturbing the reactive power setpoint and the adjustment amount is calculated to update the reactive power setpoint. This process is performed a finite number of times per time step.
[0017] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows: This invention combines droop control and improved multi-agent deep reinforcement learning (MADDPG) to generate the optimal reactive power setpoint offline, and uses an LSTM model for prediction and sensitivity-based online correction, achieving more precise voltage regulation than simple droop control. This solves the problem of insufficient voltage regulation in traditional droop control in high-penetration renewable energy distribution systems.
[0018] This invention introduces a supervised learning framework guided by offline historical data selection and physical knowledge, eliminating the need for online trial and error. Specifically, it first selects the optimal reactive power setpoint with the minimum voltage deviation from historical data through offline simulation and stores it as a knowledge base. Then, it uses an LSTM model for supervised learning (avoiding real-time reinforcement learning) to predict the optimal reactive power setpoint for the current scenario. The results are stable in multiple simulation runs, while MAPPO and MADDPG exhibit unpredictable performance fluctuations. This demonstrates that this invention provides more reliable control through deterministic offline optimization and data-driven prediction, overcoming the poor reliability and inconsistent performance problems caused by randomness in the pure multi-agent deep reinforcement learning (MADRL) method in actual voltage regulation.
[0019] This invention shifts the time-consuming learning and optimization process to an offline stage, using only lightweight LSTM prediction and sensitivity correction iterations online. This significantly reduces the computational burden during execution, eliminating the need for high-performance computing hardware. This design maintains high performance while drastically reducing computational resource requirements, improving the method's feasibility and scalability in resource-constrained environments. It also reduces the high algorithmic complexity and computational resource demands of methods like Physical Information Driven MADRL, making them easier to deploy in practice.
[0020] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0021] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart illustrating the voltage control method for a power distribution system based on optimal historical selection and prediction, according to an embodiment of the present invention. Figure 2 This is a flowchart of the OHSF scheme according to an embodiment of the present invention; Figure 3 This is a modified MADDPG schematic diagram according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a 33-node power distribution system with photovoltaic distributed energy according to an embodiment of the present invention; Figure 5 This is a graph showing the six generations of photovoltaic systems and average load curves in the history of this invention. Figure 6 This is a (real-time) photovoltaic and load curve diagram as expected in the embodiments of the present invention; Figure 7 This is a training loss curve for each round of the OHSF scheme based on LSTM, as described in an embodiment of the present invention. Figure 8 The Q-setpoints for the OHSF scheme in this embodiment of the invention are: (a) summer, (b) spring, and (c) winter scenarios. Figure 9 This is a voltage curve of the photovoltaic bus during the summer daytime according to an embodiment of the present invention; Figure 10 This is a daytime photovoltaic bus voltage curve diagram in spring according to an embodiment of the present invention; Figure 11 This is a voltage curve of a photovoltaic node under winter conditions according to an embodiment of the present invention; Figure 12 A comparison of the average voltage curves in embodiments of the present invention for daily scenarios in (a) summer, (b) spring, and (c) winter; Figure 13 This is a comparison chart of five consecutive simulation runs of an embodiment of the present invention. Detailed Implementation
[0022] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.
[0023] like Figure 1 As shown in the figure, a power distribution system voltage control method based on optimal historical selection and prediction according to an embodiment of the present invention includes steps S1-S3.
[0024] S1: Offline phase: Based on historical data, determine the optimal reactive power setpoint for each photovoltaic distributed energy node and build an optimal setpoint knowledge base.
[0025] Specifically, in the offline phase: based on historical photovoltaic power generation data and load data, corresponding reactive power setpoints are generated using rule-based droop control algorithms and improved MADDPG algorithms, respectively. The results of the rule-based droop control algorithm and the improved MADDPG algorithm are compared, and the optimal reactive power setpoint with the minimum voltage deviation is selected for each photovoltaic distributed energy node, thus constructing an optimal setpoint knowledge base.
[0026] The objective function is to minimize the deviation of all bus voltages from the reference voltage. The problem is formulated as follows: (1) Where T is the number of time steps and N is the number of buses. It is set to a reference voltage of 1.0 pu, and The voltage magnitude (in pu) at bus i at time step t is determined by power flow calculations. (2) in It is the voltage angle difference. It is the conductance and induction of the admittance matrix. The voltage amplitude of bus i. Let j be the magnitude of the bus voltage. These power flow equations are solved by the following constraints: (3) (4) in and These are the minimum and maximum voltages, respectively. m and n are the upper and lower voltage dead zones, respectively (m=0.98 and n=1.02 in this embodiment). Equation (4) is the voltage dead zone constraint, and there is no reactive power Q support between the dead zones.
[0027] Distributed energy resources (DERs) should be considered in the inverter's apparent power ( Within its capacity, the DER provides injected (lagging) and absorbed (leading) reactive power Q support. The available reactive power Q is limited by the PQ curve, meaning that as active power output P increases, the maximum reactive power Q decreases. When P output is zero, the DER can provide its maximum reactive power Q. The reactive power Q generated by the system is limited by the PQ curve, therefore the maximum reactive power Q decreases as P output increases. When P output is zero, the DER can provide its maximum reactive power Q, thus ensuring continuous voltage support under all operating conditions. Therefore, the time-varying reactive power of each photovoltaic distributed energy source... Subject to the following constraints: (5) in It is the rated apparent power. It is the active power output of the photovoltaic inverter.
[0028] In the reinforcement learning framework, the interaction between agents (PV DERs) and the environment (PDN) is formalized as a Markov game, consisting of tuples. express.
[0029] agent : The agent set i corresponding to photovoltaic distributed energy resources (DERs).
[0030] state space Each agent Observing a local state ,in It is the voltage amplitude of the proxy bus. It is the P-injection of the photovoltaic distributed generator. This is the normalized time step. The joint state is... .
[0031] Action space Each agent i selects one action. , representing reactive power injection. This action is dynamically constrained by equation (5). The combined action is .
[0032] Transition Dynamics State transition function Modeled by a power flow solver, which calculates the next state based on the current state s, the joint action a, and the time-varying PV and load distribution. Due to the nonlinear power flow equations, this transition is deterministic but complex.
[0033] reward function Each agent determines the voltage of the next state. Receive rewards : (6) Discount factor The discount factor strikes a balance between immediate and future rewards, enabling agents to make decisions that optimize short-term performance and long-term outcomes.
[0034] like Figure 2 As shown in the flowchart, the Optimal History Selection and Prediction (OHSF) scheme proposed in this invention includes three main parts: 1) Calculating the reactive power Q setpoint for voltage control through droop control and improved MADDPG based on historical photovoltaic and load data; 2) Selecting the optimal reactive power Q setpoint with the minimum voltage deviation for the photovoltaic bus; 3) Predicting the reactive power Q setpoint result corresponding to the expected (real-time) photovoltaic and load data using an LSTM network.
[0035] OHSF based on drooping: The inverter adjusts reactive power based on local voltage deviation.
[0036] (7) in, It is the droop coefficient. is the reference voltage (here, 1p.u.), and v is the photovoltaic bus voltage.
[0037] OHSF scheme based on improved MADDPG: Figure 3 An improved MADDPG algorithm is shown. This algorithm combines the actor-critic method with continuous actions and has the following main features: Furthermore, the improved MADDPG algorithm includes: Actor Network: Each photovoltaic node acts as an agent, with a policy function. For a neural network, the input is the local state. The output is reactive power operation. ; Commentator Network: Includes a centralized action-value function The input is the joint state of all agents. and combined effects The output is a Q-value, used to evaluate the state. The following joint actions were taken. The expected cumulative return; Target Network: Set up target networks for both actors and commentators. and And by updating the rules (12) Update.
[0038] Specifically, for an actor network: each agent i has a deterministic policy Parameterized as This is implemented as a neural network layer. Tanh activation ensures that the action is amplified to [-1, 1], and then... Mapped to eq(5), each actor i is updated to maximize the desired action-value function (q-value): (8) in It is the objective (loss) function of agent i's policy, measuring the current policy parameters. The degree of goodness. These are the parameters (weights) of the actor (policy) network of agent i. In the actor-critic approach, the policy is usually deterministic. It is a playback buffer that contains storage transfers. , It is the expectation of the state sampled from the replay buffer. It is the local observation (state) of agent i. It is the deterministic policy (executor) of agent i. These are the parameters (weights) of the commentator network. The actor network updates using deterministic gradients, where the gradient is: (9) in, The actor parameter represents the expected return relative to agent i. The gradient. Batch size is B, and j indexes the samples drawn from the playback buffer. Item This represents the importance sampling or weighting factor for the j-th sample. (Function) It is the deterministic strategy (actor) of agent i, and its local observation Mapped to an action. Criterion function. It is a centralized action-value function with parameters. Used to evaluate the global state The combined effect of gradients (a). The sensitivity of critics to agent i's actions is measured by an evaluation value of [value missing]. This allows the policy to be updated through deterministic gradients.
[0039] Specifically, for comment networks: a centralized critic approximation of the action-value function Take the joint state synergistic effect As input, parameterized as The critic is updated to minimize the time difference (TD) loss between the target Q-value and the predicted Q-value. As shown in equations (10) and (11): (10) (11) In equation (10), Let B represent the mean squared error loss function of the critic network, B be the mini-batch size, and j be the sample index sampled from the empirical replay pool. The weight of the j-th sample (e.g., importance sampling weight). For parameters The centralized critic network represented in the state and joint actions Estimation of the action value function under the following conditions Let Q be the target Q value. In equation (11), It consists of immediate returns and discounted future returns, of which Indicates the total number of agents. The instantaneous reward obtained by the i-th agent in the j-th sample. As a discount factor, This is a termination flag (if the state is terminated, then...). ), This represents the global state at the next moment. For the target policy network, A target commentator network is used to stabilize the training process.
[0040] Specifically, regarding the target network: each role and commentator has a target network. and its parameters and and Initialize a replica of the main network. These contents will be slightly updated: (12) in .
[0041] Furthermore, the improved MADDPG algorithm includes: Prioritize experience playback, and assign priorities based on the absolute value of the TD error of the experience samples; Adaptive noise that decays with the number of training rounds is used; Learning rate scheduling involves decaying the learning rate after a preset number of rounds.
[0042] Specifically, priority experience replay: In the replay buffer, experiences are randomly sampled during training to break the correlation between consecutive experiences and stabilize learning. Prioritized learning focuses on high-error transitions, improving the efficiency of the PDN. Transitions are sampled with the following probabilities: (13) in, It is the TD error of transition j, and l is the loop for each experience. Control the amount of priority usage.
[0043] Specifically, adaptive noise: This invention uses Gaussian noise, as follows: (14) in, It is the standard deviation of the Gaussian noise during training step t. The initial value for training / optimization is 𝜎 (between 0.5 and 1). The final (minimum) value of 𝜎 at the end of training (between 0.01 and 0.05). This is the index of the current iteration, indicating how far the algorithm has progressed. E is the frequency of noise attenuation.
[0044] Specifically, learning rate scheduling: StepLR method is used to implement stepped learning rate scheduling. For each optimizer, the learning rate decreases once every 100 epochs. The gradually decaying learning rate can stabilize training, allowing for fine-tuning of the policy in later epochs.
[0045] Furthermore, the step of selecting the optimal reactive power setpoint with the minimum voltage deviation for each photovoltaic distributed energy node includes: For each reactive power setpoint, the corresponding total squared voltage deviation is calculated, and the total squared voltage deviation is based on the formula... Confirmed, among which The voltage of bus b at hour h. Reference voltage; Compare the total square voltage deviation of all reactive power setpoints and select the reactive power setpoint with the smallest deviation as the optimal setpoint.
[0046] Specifically, by executing the improved MADDPG algorithm multiple times, the setpoint of reactive power Q is typically observed to be at... The method was subsequently extended to analyze voltage curve data from multiple simulation outputs to determine the optimal reactive power Q setpoint for photovoltaic distributed energy resources (PV DERs). Specifically, the algorithm calculates the reactive power Q setpoint of a specified PV DER relative to a reference voltage over a 24-hour period. The total square voltage deviation (1p.u.) is expressed by the following formula: (15) This refers to the voltage of bus b at hour h, extracted from the photovoltaic distributed energy (PV DER) data obtained from the droop and improved MADDPG voltage output results. The method processes all result files matching the specified naming pattern, extracts the voltage and Q control point, and identifies the reactive power Q setpoint that produces the lowest voltage deviation for each PV DER and scenario. The optimal reactive power Q setpoint is then saved to an Excel file for further analysis.
[0047] Furthermore, the optimal setpoint knowledge base includes historical photovoltaic power generation curves, corresponding load conditions, the location of each photovoltaic bus, and the corresponding optimal reactive power (Q) setpoint.
[0048] S2: At the current stage: Based on real-time data, predict the optimal reactive power setpoint for each photovoltaic distributed energy node in the current scenario.
[0049] Specifically, in the online phase: real-time photovoltaic power generation data and load data are acquired, and the closest historical scenario is matched in the knowledge base by calculating the similarity with historical photovoltaic curves. The optimal reactive power setpoint of each photovoltaic node in the current scenario is predicted using a pre-trained long short-term memory neural network model.
[0050] OHSF scheme based on LSTM: This invention combines LSTM neural networks with power system analysis to optimize the reactive power Q and voltage distribution of photovoltaic distributed energy resources (DERs) in distribution networks. By combining data-driven and physical knowledge, the algorithm aims to make short-term (10-minute interval) predictions of expected (real-time) photovoltaic and load curves and minimize voltage deviations. The algorithm executes according to the following key steps: Profile matching: for each time step (Corresponding to a time interval of 10 minutes within 24 hours), the algorithm uses absolute difference as a similarity index to identify and compare the observed photovoltaic power generation values. The closest historical photovoltaic (PV) curves. Historical curves include: PV1, PV2, PV3, PV4, PV5, and PV6, with corresponding PV data as follows: ,in The formula for calculating the distance of each curve is: (16) The most recent configuration file index is: (17) In addition, the PV difference is calculated as follows: This step-by-step matching makes it possible to construct composite historical scenarios by selecting the reactive power Q setpoint and load data corresponding to the most similar configuration file. This data is used to generate synthetic training data for the LSTM model, with a target value Q adjusted based on the PV ratio, and historical PV values that are non-zero.
[0051] For the Long Short-Term Memory (LSTM) neural network model to predict the optimal reactive power setpoint for each photovoltaic (PV) node in the current scenario: Further, the input to the LSM model is normalized time-series data, including PV power generation data and load data, where the input vector is represented as... ,in This represents the normalized total active power generation of the photovoltaic DER. is the load scaling factor for normalized time step t, corresponding to 144 time steps in 10 minutes; The output of the long short-term memory neural network model is the reactive power setpoint vector. .
[0052] Furthermore, the long short-term memory neural network model includes two stacked LSTM layers, each layer containing 64 hidden units, with dropout regularization applied between layers, and the hidden state mapped to the output through a fully connected layer; The training process uses 200 days of synthetically generated data, with each day containing 144 time steps, and the loss function is as shown in equation (20). The input and output of the Long Short-Term Memory Neural Network Model are both normalized based on the mean and standard deviation calculated from the training set.
[0053] Specifically, the algorithm uses an LSTM neural network to predict the reactive power Q setpoint of the PVDER. The model input is a normalized vector time series ( ): (18) in This refers to the active power generation of the total photovoltaic DER. It is the time step t (10-minute resolution). The load scaling factor. The target output for each step ( ) is the Q-set point vector: (19) The LSTM architecture consists of two stacked LSTM layers, each with 64 hidden units. A dropout of 0.2 is applied between layers for regularization. The final fully connected (dense) layer maps the hidden states to the output values. Training is performed over 200 synthetically generated days, each containing 144 time steps. Real-world noise is added to the observed PV and load profiles, and these are mapped to the reactive power Q setpoint (PV-to-Q ratio) of the closest historical scenario. This is achieved through [method / method]. The loss function is the mean squared error: (20) in Where n is the number of photovoltaic distributed energy sources (PV DERs). To predict the Q value of bus i on day d, at time step t, Let Q be the true value (measured or target value) at date i, time step t, and bus i. Use the Adam optimizer (learning rate 0.003) for 300 training epochs.
[0054] Input and output features are normalized using the mean and standard deviation computed over the entire training set: ;(twenty one) in, For input values, The average value of the input data x. Let x be the standard deviation of the input data. To use a smaller positive constant (e.g., 10) -8 This can prevent division by zero. This is used to normalize (standardize) the input, with a mean of zero and a variance of 1 (approximate value), and to improve numerical stability. For the final prediction output, This represents the model's predicted output in the normalized space. Let y be the standard deviation of the target variable. Let y be the mean of the target variable.
[0055] After training, the model will be applied to the normalized photovoltaic and load sequences for the expected days to generate initial reactive power Q predictions for all 144 steps. These predictions are denormalized and strictly limited to the range of Equation (1).
[0056] S3: Perform online iterative voltage correction based on power system sensitivity on the predicted optimal reactive power setpoint, and send the corrected setpoint to each photovoltaic inverter for execution to realize power distribution system voltage regulation.
[0057] Furthermore, the online iterative voltage correction based on power system sensitivity for the predicted optimal reactive power setpoint includes: For each time step, the voltage error of the photovoltaic access bus is calculated. If the absolute value of the error exceeds the dead zone threshold, the local voltage sensitivity is estimated by perturbing the reactive power setpoint and the adjustment amount is calculated to update the reactive power setpoint. This process is performed a finite number of times per time step.
[0058] Specifically, for each 10-minute time step, the predicted reactive power Q setpoint and real-time photovoltaic power generation data are used. and load Perform AC power flow calculations to obtain the voltage values of each photovoltaic bus. ; Calculate the voltage error of each photovoltaic bus k If the absolute value of the voltage error is less than or equal to the dead zone threshold, the reactive power Q setpoint of the bus is not adjusted; otherwise, iterative correction is performed: a disturbance (±0.05 Mvar) is applied to the reactive power Q setpoint, and the local voltage sensitivity is estimated through additional power flow calculation; the adjustment amount of the reactive power Q setpoint is calculated based on the sensitivity. The setpoint is limited to ±0.25 pu to ensure stability; the reactive power Q setpoint is updated and voltage-dependent capability constraints are applied to ensure that the setpoint meets the inverter operating limits (Equation 5); the correction process is repeated, with a maximum of three iterations per time step, until the voltage error converges or no further adjustment is needed.
[0059] More specifically, obtaining the initial reactive power setpoint from the LSTM model (And after clipping to satisfy equation (5),) a fast iterative correction procedure is applied every 10-minute time step to improve the voltage regulation of the PDN. In LSTM prediction and capability clipping Subsequently, a lightweight iterative correction is applied every 10 minutes to adjust the voltage of the PV connection bus toward the target. The process first uses predicted data. Observed and the load calculated proportionally Perform power flow simulation. For each PV agent k (corresponding to the bus) ), calculate voltage error ;if (Dead zone) The bus is not adjusted. Otherwise, by perturbing the reactive power Q... And solve for the other two power flows to estimate the local voltage sensitivity. Then calculate the required adjustment and limit it to... To ensure stability, the setpoint is updated. Then, a voltage-dependent capability limit is imposed on (5), with stricter restrictions, when When, reactive power Q is absorbed; when If the condition is met, biased injection occurs; otherwise, symmetric constraints apply. Then, the revised... The updated power flow is then performed. This correction loop iterates a maximum of three times per time step until no more meaningful adjustments are needed. This data- and physics-oriented approach provides a computationally lightweight method to achieve near-optimal voltage distribution while solving the full nonlinear optimization problem at each time step.
[0060] In one embodiment, the LSTM-based OHSF control flow is as follows: Data preparation and preprocessing: Acquiring real-time photovoltaic power output and load data The historical PV / load curves and reactive power Q setpoints were also included. The historical data was interpolated into 144 time steps, corresponding to a 10-minute time resolution. Synthetic data generation: By introducing noise perturbation and curve matching methods, 200 sets of daily photovoltaic and load data are generated to improve the diversity and robustness of training samples; LSTM model training: The input data and target reactive power are normalized, and the normalized data is used to train a two-layer LSTM model to learn the mapping relationship between photovoltaic output and load changes and reactive power setpoint. Reactive power prediction: real-time data Input the trained LSTM model to obtain the predicted reactive power sequence. The prediction results are then inversely normalized and their upper and lower limits are pruned according to equation (5). Time-step control initialization: For each time step According to real-time load Adjust the system load ratio and set the photovoltaic active power injection to... At the same time, the initial reactive power setpoint is set to... ; AC power flow calculation: Given the current active and reactive power setpoints, perform an AC power flow calculation to obtain the voltage distribution of each node in the system; Sensitivity-based reactive power correction: At most three iterative corrections are performed within each time step. For each photovoltaic agent k, if its grid-connected node voltage... If the deviation from the nominal value of 1.0 pu exceeds 0.015, then the voltage-reactive power sensitivity of that node should be estimated. Based on this, the restricted reactive power correction amount is calculated. The reactive power setting value was then updated. And ensure that it satisfies the constraints of equation (5); Convergence judgment and power flow recalculation: If no significant reactive power correction is generated in the current iteration, the sensitivity correction process is terminated early; otherwise, the AC power flow calculation is re-executed using the updated reactive power setpoint. Output control result: After completing the calculation for all time steps, the final corrected reactive power setpoint for each time step is obtained. This serves as the output of the LSTM-based OHSF control scheme.
[0061] In one embodiment, such as Figure 4 As shown, a 33-node medium-voltage distribution network (PDN) with a rated voltage of 12.66kV was used as the test system. Eight photovoltaic distributed energy sources (PVDERs) were integrated at nodes 7, 11, 14, 18, 21, 25, 27, and 32. Each PV unit was equipped with an inverter with a capacity overrun of 20%, therefore its maximum rated capacity ( The value is 1.2 MVA, and the corresponding maximum output power is ( The capacity is 1MW. Figure 4 The diagram shows a 33-node distribution network equipped with photovoltaic distributed energy resources.
[0062] Global Horizontal Irradiance (GHI) data for a certain city used to model photovoltaic power generation was obtained from the National Solar Radiation Database (NSRDB). Six representative days were selected from the historical dataset based on the total daily GHI value to ensure coverage of different irradiance levels. PV1 (June 14, 2020) and PV2 (May 19, 2020) represent high irradiance days, PV3 (March 6, 2020) and PV4 (August 17, 2020) represent moderate irradiance days, and PV5 (November 8, 2020) and PV6 (February 29, 2020) represent low irradiance days. Furthermore, the hourly average load demand curve was derived from historical load curves. Figure 5Six photovoltaic power generation curves and average load curves are displayed.
[0063] This embodiment considers five simulation results. Four of them are from the modified MADDPG, and one is from droop control. To determine the most effective reactive power Q setpoint, the voltage deviation at the photovoltaic bus from the reference value was calculated. The reactive power Q setpoint corresponding to the minimum voltage deviation of the PV connection bus is selected for each bus, each PV curve, and each hour using Equation (15). The results are then saved as an Excel file.
[0064] Using the optimal reactive power Q setpoint / value determined for each scenario, an LSTM algorithm combining data-driven and physical information-based approaches is applied to predict the reactive power Q setpoint for each photovoltaic distributed energy source. This process comprises three main steps: load curve matching, reactive power Q prediction, and voltage correction. Three simulation studies were conducted to evaluate the effectiveness of the proposed OHSF scheme. Figure 6 The projected (real-time) photovoltaic and load curves are displayed for three representative days, representing summer, spring / autumn, and winter scenarios, respectively.
[0065] Figure 7 The training loss of the LSTM-based OHSF algorithm is shown. The entire process takes 35 seconds to complete 300 training cycles. Therefore, the execution phase of the proposed LSTM-based OHSF algorithm is computationally lightweight and does not require repeated end-to-end retraining like the physics-based MADRL. Thus, it can be deployed without high-performance computing hardware and supports fast, localized execution under predicted (real-time) photovoltaic and load curves.
[0066] Figure 8 The proposed OHSF reactive power Q setpoints are shown under three different scenarios: summer, spring, and winter. The red dashed lines represent the Q and P absorption and injection of PV DERs over time. For the limits of Equation (1), all reactive power Q setpoints of the PV DERs should be along these lines and within the quadrants. The reactive power Q setpoints of all PV DERs are perfectly consistent with the red dashed lines and quadrants. This confirms that the proposed OHSF scheme achieves optimal reactive power Q setpoints both theoretically and practically, without overestimating or underestimating the limits of PV inverters.
[0067] Next, the novelty of the proposed OHSF scheme compared to existing literature and state-of-the-art methods is explained. Voltage regulation performance is evaluated by performing power flow calculations using the reactive power Q setpoint of the proposed OHSF scheme. To demonstrate its novelty, the proposed OHSF scheme is compared with the following methods: droop control (rule-based), optimal power flow (OPF) (optimization-based), multi-agent near-end policy optimization (MAPPO) (stochastic), MADDPG (deterministic), and unity power factor (PF) operation. Figure 9 , 10 Figure 11 shows a comparison of the photovoltaic bus voltage curves under summer, spring, and winter daytime scenarios. The OHSF scheme can achieve constant voltage regulation, keeping all photovoltaic access buses at almost 1 p.u. under all operating scenarios.
[0068] Figure 12 The comparison of average voltage curves under daily scenarios in summer, spring, and winter is shown. It can be seen that the proposed OHSF scheme provides the best voltage regulation performance, maintaining the voltage at a level closest to 1.0 pu in all scenarios. This confirms that the reactive power Q setpoint generated by the proposed OHSF scheme is accurate and effective, and can be used for voltage regulation in distribution networks with high photovoltaic distributed energy penetration over short periods (10-minute intervals).
[0069] Next, Figure 13 The results of five consecutive simulation runs are presented for the proposed OHSF scheme, droop control, OPF, MAPPO, and MADDPG. The total voltage deviation of droop control is approximately 1.4%, 1.3%, and 1.25% in summer, spring, and winter scenarios, respectively. OPF produces deviations of approximately 1.7%, 1.6%, and 1.8% under the same conditions. Droop control and OPF consistently provide the same results across all simulation runs. In contrast, the MAPPO and MADDPG methods exhibit unpredictable performance: they sometimes achieve better voltage regulation than droop control and OPF, but at other times produce worse results due to their learning-based heuristic algorithms. The proposed OHSF scheme produces a minimum voltage deviation of approximately 0.6% in summer, 0.5% in spring, and 0.4% in winter, while maintaining almost identical results across all simulation tests with minimal fluctuations (±0.02%).
[0070] Furthermore, the proposed OHSF scheme reduces algorithmic complexity and does not rely on high-performance computing hardware. Employing a data-driven and physics-guided framework, combining centralized training with distributed implementation, significantly reduces communication requirements. Moreover, the proposed OHSF scheme supports online inference and eliminates the need for repeated long-duration prediction simulations in real-time operation. Therefore, it achieves excellent voltage regulation performance in short-term (10-minute interval) predictions based on the expected (real-time) photovoltaic power generation and load curves.
[0071] Inverter-based distributed energy sources operate in the four quadrants of active and reactive power, and traditional droop control (VQ curve) is required by grid specifications. However, due to the high penetration rate of intermittent renewable energy, relying solely on droop control for voltage regulation in modern distribution systems remains insufficient. Although multi-agent deep reinforcement learning (MADRL) has become an advanced solution for coordinated voltage control, pure MADRL still suffers from inherent randomness, lack of formal performance guarantees, and vulnerability to inconsistent or unstable behavior, especially under conditions exceeding the training distribution, such as extreme weather or unknown network configurations. Physically informative multi-agent deep reinforcement learning (PI-MADRL) addresses these limitations by embedding physical laws and grid constraints into the learning process, thereby improving security, reliability, and practical applicability, although this increases algorithm complexity and computational requirements; related research continues to optimize this key trade-off. Therefore, the Optimal History Selection and Prediction (OHSF) scheme of this invention is a method to address the shortcomings of existing technologies. The main contents and beneficial effects to be protected by this invention include: Hybrid Offline-Online Voltage Control Framework: This invention proposes a hybrid voltage control framework that combines rule-based droop control, reinforcement learning-based optimization, and supervised short-term prediction. This framework separates computationally intensive learning from real-time operation, thereby reducing the online computational burden under high photovoltaic penetration while maintaining adaptive voltage regulation capabilities. Technical Effects: Achieves real-time voltage control with low computational complexity, improving system stability compared to purely learning-based or purely rule-based methods. Optimal Reactive Power Setpoint Generation Using a Dual Control Strategy: This invention introduces an offline simulation process where a rule-based voltage-reactive power (Var) droop controller and an improved multi-agent deep deterministic policy gradient (MADDPG) controller are applied to the same historical photovoltaic (PV) and load curves and run under predefined voltage constraints. The invention systematically compares the voltage deviation performance of the two methods and selects the optimal reactive power (Q) setpoint for each PV bus and each PV power generation curve. Technical Benefits: Generates a globally optimized reactive power Q setpoint without online trial and error, avoiding the stability and convergence problems of real-time reinforcement learning. A reactive power knowledge base based on load characteristics is established: the selected optimal reactive power setpoint is used as reference data storage, and the index includes: historical photovoltaic power generation curves, corresponding load conditions, and the locations of each photovoltaic bus, forming a feature-based reactive power knowledge base. Technical benefits: It enables the reuse of learned control knowledge, eliminates redundant optimization under similar operating conditions, and enhances the robustness of the system when photovoltaic fluctuation patterns change. Physically-Based Short-Term Voltage Control Supervision Learning: This invention employs a supervised LSTM model that uses real-time photovoltaic / load measurements as input and a stored optimal reactive power setpoint as a reference target guided by physical knowledge to predict the optimal reactive power command for previously unseen photovoltaic configurations over short timescales (10-minute intervals). Technical Effects: Achieves fast and stable control without reinforcement learning exploration, maintains voltage limits under unknown and rapidly changing photovoltaic conditions, and ensures compliance with inverter operating constraints. A lightweight control algorithm for power distribution systems: By transferring learning and optimization offline and using supervised inference only in the online phase, the proposed scheme constitutes a lightweight voltage control algorithm suitable for inverter-based distributed energy distribution networks. Technical benefits: Reduced communication and computing requirements; improved scalability as the number of photovoltaic units increases; and improved feasibility for practical applications compared to centralized reinforcement learning controllers.
[0072] This invention protects a hybrid voltage control method for inverter-based distributed energy, wherein an optimal reactive power setpoint is generated offline through coordinated rule-based droop control and reinforcement learning, which serves as a reference data storage dependent on the configuration file and is used by a supervised learning model to achieve fast, stable and computationally efficient voltage regulation for unknown photovoltaic power generation configuration files.
[0073] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0074] It will be readily understood by those skilled in the art that this invention includes any combination of the inventive description and specific embodiments outlined in the foregoing specification, as well as the various parts shown in the accompanying drawings. Due to space limitations and for the sake of brevity, not all of these combinations have been described in detail. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.
[0075] Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention without departing from the principles and spirit of the invention. The scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A voltage control method for a power distribution system based on optimal historical selection and prediction, characterized in that, include: Offline phase: Based on historical data, determine the optimal reactive power setpoint for each photovoltaic distributed energy node and build an optimal setpoint knowledge base; At this stage: Based on real-time data, predict the optimal reactive power setpoint for each photovoltaic distributed energy node in the current scenario; The predicted optimal reactive power setpoint is subjected to online iterative voltage correction based on sensitivity, and the corrected setpoint is sent to each photovoltaic inverter for execution to realize voltage regulation of the power distribution system.
2. The distribution system voltage control method based on optimal historical selection and prediction as described in claim 1, characterized in that, Offline phase: Based on historical photovoltaic power generation data and load data, corresponding reactive power setpoints are generated using rule-based droop control algorithms and improved MADDPG algorithms, respectively. The optimal reactive power setpoint with the minimum voltage deviation is selected for each photovoltaic distributed energy node, and an optimal setpoint knowledge base is constructed. Online phase: Acquire real-time photovoltaic power generation data and load data, match the closest historical scenario in the knowledge base by calculating the similarity with historical photovoltaic curves, and use a pre-trained long short-term memory neural network model to predict the optimal reactive power setpoint of each photovoltaic distributed energy node in the current scenario.
3. The distribution system voltage control method based on optimal historical selection and prediction as described in claim 2, characterized in that, The improved MADDPG algorithm includes: Actor Network: Each photovoltaic node acts as an agent, with a policy function. For a neural network, the input is the local state. The output is reactive power operation. ; Commentator Network: Includes a centralized action-value function The input is the joint state of all agents. and combined effects The output is a Q-value, used to evaluate the state. The following joint actions were taken. The expected cumulative return; Target Network: Set up target networks for both actors and commentators. and And by updating the rules and Update.
4. The distribution system voltage control method based on optimal historical selection and prediction as described in claim 3, characterized in that, The actor network is updated via deterministic gradients, where the gradient is: ； in, The actor parameter represents the expected return relative to agent i. The gradient is given, the batch size is B, and the index j is the sample drawn from the playback buffer. The function represents the importance sampling or weighting factor of the j-th sample. It is a deterministic strategy of agent i, which observes its local area. Mapping to an action, the critique function It is a centralized action-value function with parameters. Used to evaluate the global state The combined effect of gradients The sensitivity of critics to agent i's actions is measured by an evaluation value of [value missing]. .
5. The distribution system voltage control method based on optimal historical selection and prediction as described in claim 3, characterized in that, The commentator network minimizes the time difference loss between the target Q-value and the predicted Q-value. Implement update: in, Let B represent the mean squared error loss function of the commenter network, B be the mini-batch size, and j be the sample index sampled from the empirical replay pool. Let j be the weight of the j-th sample. For parameters The centralized critic network represented in the state and joint actions Estimation of the action value function under the following conditions For the target Q value, It consists of immediate returns and discounted future returns. Indicates the total number of agents. The instantaneous reward obtained by the i-th agent in the j-th sample. As a discount factor, As a termination marker, This represents the global state at the next moment. For the target policy network, For the target commentator network.
6. The distribution system voltage control method based on optimal historical selection and prediction as described in any one of claims 2-4, characterized in that, The improved MADDPG algorithm includes: Prioritize experience playback, and assign priorities based on the absolute value of the TD error of the experience samples; Adaptive noise that decays with the number of training rounds is used; Learning rate scheduling involves decaying the learning rate after a preset number of rounds.
7. The distribution system voltage control method based on optimal historical selection and prediction as described in claim 2, characterized in that, The selection of the optimal reactive power setpoint with the minimum voltage deviation for each photovoltaic distributed energy node includes: For each reactive power setpoint, the corresponding total squared voltage deviation is calculated, and the total squared voltage deviation is based on the formula... Confirmed, among which The voltage of bus b at hour h. Reference voltage; Compare the total square voltage deviation of all reactive power setpoints and select the reactive power setpoint with the smallest deviation as the optimal setpoint.
8. The distribution system voltage control method based on optimal historical selection and prediction as described in claim 2, characterized in that, The input to the Long Short-Term Memory (LSTM) neural network model is normalized time-series data, including photovoltaic power generation data and load data, where the input vector is represented as... ,in This represents the total active power generation of the photovoltaic DER. t is the load scaling factor for time step t, corresponding to 144 time steps in 10 minutes; The output of the long short-term memory neural network model is the reactive power setpoint vector. .
9. The distribution system voltage control method based on optimal historical selection and prediction as described in claim 2 or 8, characterized in that, The Long Short-Term Memory Neural Network Model includes two stacked LSTM layers, each containing 64 hidden units. Dropout regularization is applied between the layers, and the hidden states are mapped to the output through a fully connected layer. The training process uses 200 days of synthetically generated data, with each day containing 144 time steps. The loss function is as follows: ； in, , where n is the number of photovoltaic distributed energy sources; the Adam optimizer is used with a learning rate of 0.003, and a total of 300 training cycles are performed; The input and output of the Long Short-Term Memory Neural Network Model are both normalized based on the mean and standard deviation calculated from the training set.
10. The distribution system voltage control method based on optimal historical selection and prediction as described in claim 1, characterized in that, The online iterative voltage correction based on sensitivity for the predicted optimal reactive power setpoint includes: For each time step, the voltage error of the photovoltaic access bus is calculated. If the absolute value of the error exceeds the dead zone threshold, the local voltage sensitivity is estimated by perturbing the reactive power setpoint and the adjustment amount is calculated to update the reactive power setpoint. This process is performed a finite number of times per time step.