Power distribution network voltage model prediction control method, device, equipment and medium

Abstract

The invention belongs to the technical field of intelligent power grids, and particularly relates to a power distribution network voltage model prediction control method and device, equipment and a medium. The control method comprises the following steps: acquiring a real-time power grid state and a prediction state in an observation scale which takes a current moment as a starting point and has a length of d; the voltage control device is input to a pre-trained SAC-MPC intelligent agent, a control action within an action scale with the length of 1 is output, and the action is a reactive power output instruction assigned to the voltage control device; and issuing and executing the control action of the next time step. According to the training method, a high-fidelity power distribution network operation environment is constructed, and based on an SAC algorithm framework, an intelligent agent is trained through environment interaction, experience playback and cooperative update of an Actor strategy network and Critic evaluation network parameters. Through combination of prospective prediction and real-time decision, the hysteresis problem of a traditional control method is solved, and rapid and smooth control of the voltage is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of smart grid, and particularly relates to a power distribution network voltage model predictive control method, device, equipment and medium. BACKGROUND

[0002] With large-scale access of distributed power generation (DG) to the power distribution network, the intermittency of the DG output and the volatility of the load are superimposed, leading to frequent source-load dynamic imbalance, causing node voltage out-of-limit, power quality decline and other problems. Although the traditional scheme relying on energy storage system to suppress power fluctuation can be short-term relief, it is difficult to cope with the complex supply-demand imbalance scene under the large-scale DG grid connection due to the defects of high investment cost, physical space constraints and insufficient flexibility. Therefore, the system operation needs to be actively regulated and optimized, and the core contradiction is that on the one hand, the regulation system requires high time resolution (seconds to minutes) dynamic perception and response capability to quickly track the source-load changes, and on the other hand, it needs to avoid the mechanical loss and control instability risk caused by frequent adjustment of devices (such as inverters and reactive power compensation devices). SUMMARY

[0003] The purpose of the present application is to provide a power distribution network voltage model predictive control method, device, equipment and medium to at least solve or improve one of the problems in the prior art.

[0004] In order to achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows: The first aspect of the present application provides a power distribution network voltage model predictive control method, comprising: obtaining real-time grid states and predicted states of the power distribution network within an observation scale; wherein the observation scale is a time interval with the current time t0 as the starting point and a length of d; the real-time grid states and the predicted states include node voltage, distributed power generation output, load power and line active power; inputting the real-time grid states and the predicted states into a pre-trained SAC-MPC agent, and outputting control actions within an action scale by the SAC-MPC agent; wherein the action scale is a time interval with the current time t0 as the starting point and a length of l ; and l ≤d; the control action is a reactive power output instruction assigned to the voltage control device; issuing the control action to the corresponding voltage control device in the power distribution network, wherein only the control action of the next time step is issued to the voltage control device; obtaining the grid states after the voltage control device executes the control action and makes the state of the power distribution network environment change, and feeding back to the SAC-MPC agent for subsequent control.

[0005] Furthermore, the training method for SAC-MPC agents includes: Create a distribution network operating environment. In each step, the environment modifies the output power of the controllable units in the grid based on the actions of all agents, and then performs power flow calculations to update the grid state based on the time-varying patterns of distributed power sources and load power. Construct a SAC-MPC agent, which includes an Actor policy network, a Critic evaluation network, and a corresponding target network; The system operates within the power distribution network environment, enabling the intelligent agent to interact with the environment and storing the interaction data in the experience replay pool. The interaction data includes state, action, reward, and next state. Sample batch interaction data from the experience replay pool and perform the following network update operations: Based on batch interactive data, the parameters of the Critic network are updated by minimizing the Bellman error of the Q function; Based on the updated Critic network parameters, the parameters of the Target Critic network are updated using a soft update method; Based on batch interaction data, the parameters of the Actor policy network are updated by maximizing policy entropy and Q-value.

[0006] Furthermore, the reward function of the SAC-MPC agent includes at least a weighted sum of voltage over-limit penalty terms, power loss penalty terms in the distribution network, and active power output reduction penalty terms in the distributed generation.

[0007] Furthermore, voltage over-limit penalty item Represented as:

[0008] in, For reference voltage, For the maximum allowable deviation, for t Time of the first i The per-unit voltage value of each node. N This represents the total number of nodes in the power grid system. Line loss bonus Represented as:

[0009] in, Q i,t PV and Q i,t SVC It is a control variable. Q i,t PV Represents a node i At any momentt The reactive power of the photovoltaic inverter; Q i,t SVC Represents a node i At any moment t The reactive power of the SVC; V 0 indicates the rated voltage; g ij Here is the admittance matrix; V j,t For node at time t j The voltage amplitude; θ ij,t for i , j Node voltage phase angle difference; β It is a weighting coefficient used to balance the two objectives of voltage deviation and network loss; Distributed power generation economic penalty Represented as:

[0010] in, N DG A collection of distributed power sources in the power grid. Distributed power sources i exist t The maximum active power output and the actual active power output at any given time.

[0011] Furthermore, the objective function of the updated Actor policy network is to maximize the sum of expected reward and policy entropy.

[0012] Furthermore, the parameters of the Critic network are updated by minimizing the Bellman error of the Q function, as follows:

[0013]

[0014] in, For the first i indivual Q Error-weighted average of the network; To evaluate the network parameters; Indicates in Below Q The network error is weighted and averaged. and They respectively represent t Time-state variable set Below, in ( t 0, l Actions to be performed during a specific time period The iThe target critic and the critic network are given. Q value; γ for Q The discount factor of the function; Represents the network parameters in the policy network. Strategies under certain conditions; Indicates the action The reward value, α represents the reward value based on the strategy. The output actions of each distributed power source are represented by R, which is the reward calculated by the reward formula based on the feedback from the power grid after the action α is executed.

[0015] Furthermore, the parameters of the Actor policy network are updated by maximizing the policy entropy and Q-value, as follows:

[0016] in, Let the loss function be the policy network. For the network parameters of the policy network, [ ] indicates in the set of state variables Weighted average; [ [For strategy] π Below, actions based on strategy selection Expected value For the set of state variables Next, strategy π Select Action The probability of.

[0017] A third aspect of the present invention provides a voltage model prediction and control device for a distribution network, comprising: The distribution network status acquisition module is used to acquire the real-time and predicted status of the distribution network within the observation scale. The observation scale is a time interval with a length of d starting from the current time t0. The real-time and predicted status of the distribution network includes node voltage, distributed generation output, load power, and line active power. The control action generation module is used to input the real-time power grid state and the predicted state into a pre-trained SAC-MPC agent, which then outputs control actions within an action scale. The action scale is defined as starting from the current time t0 and having a length of... l The time interval, and l ≤d; Control action refers to the reactive power output command assigned to the voltage control equipment; The instruction issuing module is used to issue control actions to the corresponding voltage control equipment in the distribution network. Specifically, only the control actions for the next time step are issued to the voltage control equipment. The feedback module is used to obtain the power grid status after the voltage control equipment performs control actions, causing changes in the power distribution network environment, and feeds it back to the SAC-MPC intelligent agent for subsequent control.

[0018] In a third aspect, the present invention provides an electronic device including a processor and a memory, the processor being configured to execute a computer program stored in the memory to implement the distribution network voltage model predictive control method as described above.

[0019] In a fourth aspect, the present invention provides a computer-readable storage medium storing at least one instruction that, when executed by a processor, implements the distribution network voltage model predictive control method as described above.

[0020] Compared with existing technologies, this invention provides a deep reinforcement learning-based model predictive control method for distribution network voltage, optimizing the problem of voltage regulation difficulties arising from the large-scale integration of new energy sources into the distribution network. Specifically, in the reinforcement learning part, this invention adopts the soft actor-critic reinforcement learning (SAC) framework and integrates model predictive control ideas into the SAC-based distribution network voltage control model, forming a distribution network voltage model predictive control agent based on soft actor-critic reinforcement learning (SAC-MPC agent). Unlike traditional reinforcement learning voltage control, the SAC-MPC agent obtains distributed power output, load, and distribution network state estimation information for the next few control steps from the distribution network prediction system. Combining this with historical distribution network operating data, the SAC-MPC agent provides a unified control strategy for the next few control steps, enabling the control strategy to achieve optimal control performance in the future time period and reducing frequent adjustments to voltage control equipment, thus achieving optimized control of the distribution network voltage. This solves the shortcomings of traditional deep learning methods in handling complex distribution network states. The SAC algorithm enhances the system's exploration capability and stability by optimizing policy entropy and expected reward, thereby improving the accuracy and robustness of voltage control and providing a more efficient solution for distribution network voltage control. Attached Figure Description

[0021] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart of a power distribution network voltage model prediction and control method according to an embodiment of the present invention; Figure 2 This is a diagram of a multi-agent voltage control system for a power distribution network in an embodiment of the present invention; Figure 3 This is a diagram of the SAC-MPC agent reinforcement learning training and execution architecture in an embodiment of the present invention; Figure 4 This is a structural block diagram of a power distribution network voltage model prediction and control device according to an embodiment of the present invention; Figure 5 This is a structural block diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0022] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0023] The following detailed description is exemplary and intended to provide further detailed explanation of the invention. Unless otherwise specified, all technical terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used in this invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention.

[0024] Example 1 like Figure 1 As shown, a voltage model predictive control method for a distribution network includes: S1. Obtain the real-time and predicted power grid status of the distribution network within the observation scale; wherein, the observation scale is a time interval with a length of d starting from the current time t0; the real-time and predicted power grid status includes node voltage, distributed generation output, load power, and line active power; S2. Input the real-time power grid state and the predicted state into the pre-trained SAC-MPC agent, and the SAC-MPC agent outputs control actions within the action scale; wherein, the action scale is a length of [value missing] starting from the current time t0. l The time interval, and l ≤d; Control action refers to the reactive power output command assigned to the voltage control equipment; S3. Send the control action to the corresponding voltage control equipment in the distribution network, wherein only the control action of the next time step is sent to the voltage control equipment; S4. Obtain the power grid status after the voltage control equipment executes control actions that change the state of the power distribution network environment, and feed it back to the SAC-MPC intelligent agent for subsequent control.

[0025] Traditional distribution network voltage control methods have three main problems when facing the access of distributed power sources with high proportion and strong fluctuations: (1) Control lag: Feedback control based on the current instantaneous state is difficult to cope with the rapid changes of source and load, resulting in slow adjustment response and voltage over-limit; (2) Frequent control and oscillation: The issuance of instructions in a single time step may cause the control equipment to operate frequently, affecting the equipment life and system stability; (3) Poor economy: Control strategies often only focus on voltage stability, ignoring line losses and the economic benefits of distributed power sources themselves, resulting in low overall energy efficiency.

[0026] This invention requires the input of the intelligent agent to be state and prediction data within an "observation scale" of length d. This enables decision-making not only based on the current state but also incorporating ultra-short-term predictions of future source-load change trends. Based on this, the intelligent agent outputs a control sequence within an "action scale" of length l, achieving "looking forward multiple steps and making multiple decisions," essentially transforming passive response into proactive forward-looking regulation, and reducing the risk of voltage exceedances caused by fluctuations in new energy sources and load.

[0027] This invention "only sends the control action for the next time step to the voltage control equipment" and incorporates the rolling optimization concept of Model Predictive Control (MPC). This means that in each control cycle, the system re-optimizes multiple steps based on the latest grid state, but only executes the first step. This ensures that the control commands can respond to the latest changes in the system in real time, and also enables smooth transitions between action commands across multiple time steps, avoiding drastic jumps in control commands. This effectively reduces the frequency of operation and mechanical stress on voltage control equipment (such as inverters and SVCs), extending the equipment's service life.

[0028] This invention employs a trained SAC-MPC agent as the decision-making core, which possesses the ability to autonomously learn optimal strategies from high-dimensional, nonlinear power grid environments. The agent can comprehensively consider multiple objectives such as voltage safety, network loss economy, and distributed power generation revenue, outputting globally optimized collaborative control commands. This overcomes the limitations of traditional methods that rely on precise mathematical models and manual parameter tuning based on experience, significantly improving the intelligence level and overall economy of voltage control in complex distribution network environments.

[0029] In one embodiment, the training method for the SAC-MPC agent includes: S21. Create a distribution network operating environment. In each step, the environment modifies the output power of the controllable units in the power grid according to the actions of all agents, and then performs power flow calculations to update the power grid state according to the time-varying laws of distributed power sources and load power. S22. Construct a SAC-MPC agent, which includes an Actor policy network, a Critic evaluation network, and a corresponding target network. S23. Run the power distribution network operating environment, enabling the intelligent agent to interact with the environment and storing the interaction data in the experience replay pool; the interaction data includes state, action, reward and next state; S24. Sample batch interaction data from the experience replay pool and perform the following network update operations: Based on batch interactive data, the parameters of the Critic network are updated by minimizing the Bellman error of the Q function; Based on the updated Critic network parameters, the parameters of the Target Critic network are updated using a soft update method; Based on batch interaction data, the parameters of the Actor policy network are updated by maximizing policy entropy and Q-value.

[0030] The training quality of deep reinforcement learning agents directly determines their performance in actual control. Traditional training methods face the following challenges: (1) Insufficient environmental realism: Simplified simulation environments cannot accurately reflect the dynamic response of real power grids, resulting in poor generalization ability of trained agents; (2) Low training stability and efficiency: Inappropriate network parameter update strategies can easily lead to divergence or getting trapped in local optima during the training process, making it difficult to obtain a robust optimal strategy.

[0031] Steps S21 to S24 provide training methods for reinforcement learning agents, providing high-quality and highly reliable agents for the core control methods.

[0032] This scheme explicitly defines the construction method of the training environment: "At each running step, the environment modifies the output power of controllable units in the power grid based on the actions of all agents, and then updates the power grid state by performing power flow calculations based on the time-varying laws of distributed power sources and load power." This simulates the physical laws and time-varying characteristics of a real power grid, enabling the agents to learn control strategies that conform to the dynamic processes of actual power systems through interaction with the environment. This high-precision simulation based on power flow calculation is crucial to ensuring that the trained agents can be effectively applied to real-world engineering scenarios, greatly enhancing the practicality of the scheme.

[0033] This scheme employs the SAC algorithm framework, achieving an optimal balance between exploration and exploitation, thus enhancing the robustness of the policy. It utilizes the Soft Actor-Critic (SAC) maximum entropy reinforcement learning algorithm, whose objective function maximizes both expected reward and policy entropy. This encourages the agent to not only pursue high rewards during training but also to explore, preventing premature convergence to a suboptimal policy. The agent trained using this method maintains more stable and robust control performance when facing uncertainties in power grid operating conditions (such as prediction errors and load fluctuations).

[0034] This scheme ensures stable convergence during training by using a separate network structure and a soft update mechanism. Specifically, a complete network system including the Actor, Critic, and target network is constructed, and the target network parameters are updated using a "soft update" method. This effectively severs the correlation between the Critic network's estimate and the target value, breaking the temporal correlation between data. It acts like a "stabilizer" for the training process, improving the stability of the deep neural network training process, avoiding divergence in Q-value estimation, and thus ensuring that a convergent and high-performance agent model is ultimately obtained.

[0035] Specifically, the reward function of the SAC-MPC agent includes at least a weighted sum of voltage over-limit penalty terms, power loss penalty terms in the distribution network, and active power output reduction penalty terms in the distributed generation.

[0036] As a concrete example, voltage over-limit penalty Represented as:

[0037] in, For reference voltage, For the maximum allowable deviation, for t Time of the first i The per-unit voltage value of each node. N This represents the total number of nodes in the power grid system. As a concrete example, line loss bonus items Represented as:

[0038] in, Q i,t PV and Q i,t SVC It is a control variable. Q i,t PV Represents a node i At any moment t The reactive power of the photovoltaic inverter; Q i,t SVC Represents a node i At any moment t The reactive power of the SVC; V 0 indicates the rated voltage; g ij Here is the admittance matrix; V j,t For node at time t j The voltage amplitude; θ ij,t fori , j Node voltage phase angle difference; β It is a weighting coefficient used to balance the two objectives of voltage deviation and network loss; As a concrete example, the economic penalty for distributed power generation Represented as:

[0039] in, N DG A collection of distributed power sources in the power grid. Distributed power sources i exist t The maximum active power output and the actual active power output at any given time.

[0040] Specifically, the objective function for updating the Actor policy network is to maximize the sum of expected reward and policy entropy.

[0041] As a concrete example, updating the parameters of the Critic network by minimizing the Bellman error of the Q function can be expressed as:

[0042]

[0043] in, For the first i indivual Q Error-weighted average of the network; To evaluate the network parameters; Indicates in Below Q The network error is weighted and averaged. and They respectively represent t Time-state variable set Below, in ( t 0, l Actions to be performed during a specific time period The i The target critic and the critic network are given. Q value; γ for Q The discount factor of the function; Represents the network parameters in the policy network. Strategies under certain conditions; Indicates the action The reward value, α represents the reward value based on the strategy. The output actions of each distributed power source are represented by R, which is the reward calculated by the reward formula based on the feedback from the power grid after the action α is executed.

[0044] As a concrete example, updating the parameters of the Actor policy network by maximizing the policy entropy and Q-value can be expressed as:

[0045] in, Let the loss function be the policy network. For the network parameters of the policy network, [ ] indicates in the set of state variables Weighted average; [ [For strategy] π Below, actions based on strategy selection Expected value For the set of state variables Next, strategy π Select Action The probability of.

[0046] In some optional embodiments, a distribution network voltage model predictive control method is also provided, comprising the following steps: A. Construct a power distribution network simulation environment to provide an interactive environment for the intelligent agents. Based on the structure and parameters of the created power distribution network, the simulation environment provides the intelligent agents with various attributes and state values ​​of the power grid for decision-making reference. At each running step, the environment modifies the output power of controllable units in the power grid according to the actions of all intelligent agents, and then performs power flow calculations to update the power grid state based on the time-varying patterns of distributed power sources and load power.

[0047] B. Constructing the SAC-MPC intelligent agent. The basic elements of the intelligent agent system include environment, state, observation, action, and reward function. Unlike traditional intelligent agents, the SAC-MPC intelligent agent uses multiple power grid operation data and prediction data within the observation scale range as the basis for the agent's actions. The SAC-MPC intelligent agent provides actions at multiple time steps within the action scale range, that is, the output power decision for the equipment controlled by the distribution network.

[0048] C. Run the power system simulation environment and collect the initial operating dataset of the power system. Train the SAC-MPC decision agent based on the power grid simulation data. The SAC-MPC agent uses multiple power grid operating data and prediction data within the observation scale range as the basis for the agent's actions. Based on the given observations, the SAC-MPC agent simultaneously provides actions at multiple time steps within the action scale range, i.e., output power decisions for the equipment controlled by the distribution network. After obtaining all the agent's actions, the actions for the next few time steps are sent to the equipment controlled by the distribution network. By integrating all information of the distribution network within the action scale, a comprehensive reward is given to the agent's actions.

[0049] D. The trained agent provides strategies for power grid control. The agent uses observation information from multiple time steps within the observation scale as the observation dataset and provides actions for multiple time steps within the action scale. The distribution network operator only sends the actions in the next time step to the controlled equipment in the distribution network, thereby achieving fast and smooth control of the distribution network.

[0050] Furthermore, in step B, a SAC-MPC control agent is constructed. This agent acts as both a decision-maker and a learner, interacting with the environment to gain experience and continuously learning from it to obtain the optimal strategy.

[0051] Construct a global state space. The state reflects the operating state of the power system at a certain moment. The voltage of each node in the power grid, the output of distributed generation, and the load are used as the current system characteristics.

[0052] An observation space is constructed for the intelligent agent. The observations reflect the operating status of the distribution network over a specific period. Observations are based on grid node voltages, distributed generation output, load, and active power of transmission lines. For example... Figure 2 As shown, the agent's observations include the current time point. t 0 of d The power grid operating status observation over a time period of length is represented as , d The observation scale.

[0053] Constructing an action space for the intelligent agent. An agent's action refers to the command issued to voltage control equipment within a given zone to control its output power. Voltage control equipment regulates the voltage of the distribution network by outputting reactive power, and mainly includes distributed power sources (such as photovoltaic power plants, wind power plants, etc.) and static var compensators (SVCs). The control actions given by the intelligent agent are based on the current time point. t Starting from 0, with a length of l The output power of each voltage control device during the time period is expressed as: , l For the scale of action, l ≤d Both end at the same point in time.

[0054] The reward function is calculated by the following formula:

[0055] in( t 0, d The time interval, defined as the observation scale, is a period that includes... t Time 0 and length is d Time interval; t 0, l () represents the time interval for the action scale; This is the penalty value for voltage exceeding the limit. Penalty for power loss in the distribution network; To reduce active power output penalty for distributed power sources , , and represent the weights of the corresponding sub-items, respectively; For decision-making actions within the action scale, that is... t Starting from time 0 and with a length of l Unified decision-making actions within a specific timeframe; For inclusion t The length of time 0 is d The set of distribution network state variables within a certain time period.

[0056] The voltage over-limit penalty value represents the penalty for a node voltage exceeding the allowable range, and the calculation formula is as follows:

[0057] in, For reference voltage, For the maximum allowable deviation, for t Time of the first i The per-unit voltage value of each node. N This represents the number of nodes.

[0058] Line loss incentive: A negative incentive is given based on the power grid line loss.

[0059]

[0060] in, N It is the total number of nodes in the power grid system; Q i,t PV and Q i,t SVC It is a control variable. Q i,t PV Represents a nodei At any moment t The reactive power of the photovoltaic inverter; Q i,t SVC Represents a node i At any moment t The reactive power of the SVC; V 0 indicates the rated voltage. g ij Here is the admittance matrix; V j,t For node at time t j The voltage amplitude; θ ij,t for i , j Node voltage phase angle difference; β It is a weighting coefficient used to balance the two objectives of voltage deviation and network loss.

[0061] Economic Penalty for Distributed Power Sources: An agent can enhance the reactive power output capability of a distributed power source by reducing its active power output, but this strategy will affect the economic income of the distributed power source from grid connection through active power. A negative reward is given based on the reduction in the active power output of the distributed power source (the difference between the maximum active power output and the actual active power output at the current moment). Clearly, the greater the reduction in active power output, the greater the penalty.

[0062]

[0063] in N DG A collection of distributed power sources in the power grid. Distributed power sources i exist t The maximum active power output and the actual active power output at any given time.

[0064] like Figure 3 As shown, the SAC-MPC agent consists of one actor network, two critic networks, and two target critic networks. During the exploration process, experience is stored in the reply buffer.

[0065] Further, in step C, the SAC algorithm includes: The policy optimization objectives of C1 and SAC algorithms are to maximize expected reward and policy entropy.

[0066] in, Here is the entropy regularization coefficient. For policy entropy, For the reward function; It is a strategy in reinforcement learning, used to represent the rules for taking actions on the distribution network; Indicates the state variable set The probability distribution of the next action selection; τ ∈(0,1) is the discount factor for expected returns, ensuring the decay of long-term rewards; Indicates policy-based π Expected value operation; Representation strategy The objective function.

[0067] C2. Experience playback: Stores the interaction data between the agent and the environment. The interaction data includes the state, actions, and rewards of the environment. Specifically, the system stores the interaction data between the agent and its environment, including environmental feedback on states, actions, and rewards. During training, the agent interacts with the power grid environment, collecting information such as states, actions, and rewards from the environment; this information forms experience learning samples. Within a certain training period, the agent stores these experience learning samples in an experience replay pool and randomly samples from it (i.e., retrieves samples from the storage pool for learning) to increase sample diversity and avoid training bias during reinforcement learning.

[0068] C3. Update the critic network by minimizing... Q Bellman error update of the function i One critic network:

[0069]

[0070] in, γ for Q The discount factor of the function and They respectively represent t Time-state variable set Below, in ( t 0, l Actions to be performed during a specific time period The i The target critic and the critic network are given. Q The value, and so on, reflects the expected reward obtained after taking the action. For the first i indivual Q Error-weighted average of the network; To evaluate the network parameters; Indicates in Below Q The network error is weighted and averaged.

[0071] C4. Update the policy network by maximizing the policy entropy. Q Value update strategy network:

[0072] in, Let the loss function be the policy network. For the network parameters of the policy network, [ ] indicates in the set of state variables Weighted average; [ [For strategy] π Below, actions based on strategy selection Expected value For the set of state variables Next, strategy π Select Action The probability of.

[0073] C5, Target Critics Network Soft Update

[0074] in, ε This represents the soft update coefficient.

[0075] C6. Repeat C2~C5 until the requirement for reward stability is met.

[0076] Furthermore, in step D, the trained SAC-MPC model is embedded into the voltage control system of the distribution network, enabling the SAC-MPC model to interact with the distribution network environment to form action strategies, including the following steps: D1. Input the power grid state and predicted state within the observation scale into the trained SAC-MPC agent. The agent generates action strategies within the action time scale and issues action commands to the distributed power sources and reactive power compensators of the distribution network.

[0077] D2, reactive power compensators and distributed generation execute the control actions issued, causing changes in the actual distribution network environment, and feeding back the changed network state to the trained SAC-MPC model.

[0078] D3. Store the distribution network operation data with lower and higher rewards and the rewards for actions taken during the operation into a buffer pool, and update and optimize the agent network regularly.

[0079] This invention constructs a collaborative mechanism of "forward-looking prediction, real-time decision-making, and smooth execution." Specifically, it predicts the source-load evolution trend through high-precision ultra-short-term prediction technology, reducing control lag; it adopts a deep flexible strategy-evaluation reinforcement learning model to adaptively learn the optimal control strategy in complex environments, overcoming the bottlenecks of traditional models' strong dependence and low online computation efficiency; simultaneously, it introduces a model predictive control framework to continuously optimize control commands across multiple time scales, achieving smooth switching and global optimization under dynamic constraints. The integration of these two approaches balances real-time performance, robustness, and economy, providing theoretical and technical support for voltage / power coordinated control in high-proportion renewable energy distribution networks, meeting the development needs of intelligent and flexible new power systems. The SAC model employed in this invention enables the agent to balance exploration and utilization during the optimization process, improving the system's robustness, especially in the face of uncertainties in power grid operating conditions, maintaining relatively stable control performance. Through real-time interactive feedback with the distribution network environment, the agent can dynamically adjust and optimize control decisions based on immediate rewards and new states. This adaptive learning process allows the agent to adjust control strategies in real time under constantly changing power grid operating conditions, ensuring optimal voltage regulation and further guaranteeing the stable operation of the power grid.

[0080] Example 2 like Figure 4 As shown, based on the same inventive concept as the above embodiments, the present invention also provides a distribution network voltage model prediction and control device, comprising: The distribution network status acquisition module is used to acquire the real-time and predicted status of the distribution network within the observation scale. The observation scale is a time interval with a length of d starting from the current time t0. The real-time and predicted status of the distribution network includes node voltage, distributed generation output, load power, and line active power. The control action generation module is used to input the real-time power grid state and the predicted state into a pre-trained SAC-MPC agent, which then outputs control actions within an action scale. The action scale is defined as starting from the current time t0 and having a length of... l The time interval, and l ≤d; Control action refers to the reactive power output command assigned to the voltage control equipment; The instruction issuing module is used to issue control actions to the corresponding voltage control equipment in the distribution network. Specifically, only the control actions for the next time step are issued to the voltage control equipment. The feedback module is used to obtain the power grid status after the voltage control equipment performs control actions, causing changes in the power distribution network environment, and feeds it back to the SAC-MPC intelligent agent for subsequent control.

[0081] Example 3 like Figure 5 As shown, the present invention also provides an electronic device 100 for implementing a voltage model predictive control method for a power distribution network; The electronic device 100 includes a memory 101, at least one processor 102, a computer program 103 stored in the memory 101 and executable on at least one processor 102, and at least one communication bus 104.

[0082] The memory 101 can be used to store the computer program 103. The processor 102 implements the steps of the distribution network voltage model prediction control method of Embodiment 1 by running or executing the computer program stored in the memory 101 and calling the data stored in the memory 101.

[0083] The memory 101 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the electronic device 100 (such as audio data), etc. In addition, the memory 101 may include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0084] At least one processor 102 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 102 may be a microprocessor or any conventional processor. Processor 102 is the control center of electronic device 100, connecting various parts of electronic device 100 via various interfaces and lines.

[0085] The memory 101 in the electronic device 100 stores multiple instructions to implement a power distribution network voltage model predictive control method, and the processor 102 can execute multiple instructions to achieve the following: The system acquires the real-time and predicted states of the distribution network within the observation scale. The observation scale is a time interval of length d starting from the current time t0. The real-time and predicted states of the distribution network include node voltage, distributed generation output, load power, and line active power. The real-time grid state and the predicted state are input into a pre-trained SAC-MPC agent, which then outputs control actions within the action scale. The action scale is defined as a period starting from the current time t0 and having a length of... l The time interval, and l ≤d; Control action refers to the reactive power output command assigned to the voltage control equipment; The control actions are sent to the corresponding voltage control equipment in the distribution network, wherein only the control actions for the next time step are sent to the voltage control equipment; The system acquires the power grid status after the voltage control equipment executes control actions that change the state of the distribution network environment, and feeds this information back to the SAC-MPC intelligent agent for subsequent control.

[0086] Example 4 If the modules / units integrated in the electronic device 100 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, and read-only memory (ROM).

[0087] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0088] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0089] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, causing the instructions stored in the computer-readable storage medium to produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0090] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0091] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A power distribution network voltage model predictive control method, characterized in that, The method comprises the following steps: obtaining real-time power grid states and predicted states of the power distribution network within an observation scale, wherein the observation scale is a time interval with a current time t0 as a starting point and a length d; the real-time power grid states and the predicted states comprise node voltages, distributed power outputs, load powers and line active powers; The real-time power grid state and the predicted state are input to the pre-trained SAC-MPC agent, and a control action in an action scale is output by the SAC-MPC agent; wherein the action scale is a time interval with the current time t0 as a starting point and a length of l , and l ≤d; the control action is a reactive power output instruction assigned to the voltage control device. issuing control actions to corresponding voltage control devices in the power distribution network, wherein only the control action of the next time step is issued to the voltage control devices; obtaining the power grid states after the voltage control devices execute the control actions and change the states of the power distribution network environment, and feeding back to the SAC-MPC agent for subsequent control.

2. The method of claim 1, wherein, The training method of the SAC-MPC agent comprises the following steps: creating a power distribution network running environment, wherein every time a step is run, the environment modifies the output power of the controllable units in the power grid according to the actions of all agents, and then performs power flow calculation to update the power grid states according to the time-varying law of the distributed power and the load power; constructing the SAC-MPC agent, which comprises an Actor policy network, a Critic evaluation network and a corresponding target network; running the power distribution network running environment to make the agent interact with the environment, and storing the interaction data in an experience replay pool; the interaction data comprises states, actions, rewards and next states; sampling a batch of interaction data from the experience replay pool, and performing the following network update operations: updating the parameters of the Critic network by minimizing the Bellman error of the Q function based on the batch of interaction data; updating the parameters of the Target Critic network in a soft update manner based on the updated Critic network parameters; updating the parameters of the Actor policy network by maximizing the policy entropy and the Q value based on the batch of interaction data.

3. The method of claim 2, wherein, The reward function of the SAC-MPC agent at least comprises a weighted sum of a voltage out-of-limit penalty term, a power distribution network power loss penalty term and a distributed power active output reduction penalty term.

4. The method of claim 3, wherein Voltage excursion penalty term is represented as: wherein, is a reference voltage, is a maximum allowed deviation, is t the voltage unit of the i node at the moment, N is the total number of nodes in the grid system; Line loss reward term is represented as: wherein, Q i,t PV and Q i,t SVC is a control variable, Q i,t PV denotes a node i the reactive power of the photovoltaic inverter at time t ; Q i,t SVC denotes a node i the reactive power of the SVC at time t ; V 0 denotes a rated voltage; g ij is an admittance matrix; V j,t is the voltage amplitude of the node j at time t; θ ij,t is the voltage amplitude of the node i , j the phase angle difference of the node voltage; β is a weight coefficient used to balance the two objectives of voltage deviation and network loss. Distributed power supply economy penalty term is expressed as: wherein, N DG for a set of distributed power sources in an electrical grid, representing a distributed power source i At t the maximum active power output and the actual active power output at the instant of time.

5. The method of claim 2, wherein The target function of the Actor policy network is to maximize the sum of the expected return and the policy entropy.

6. The method of claim 2, wherein The parameters of the Critic network are updated by minimizing the Bellman error of the Q function, which is represented as: in, For the first i indivual Q Error-weighted average of the network; To evaluate the network parameters; Indicates in Below Q The network error is weighted and averaged. and They respectively represent t Time-state variable set Below, in ( t 0, l Actions to be performed during a specific time period The i The target critic and the critic network are given. Q value; γ for Q The discount factor of the function; Represents the network parameters in the policy network. Strategies under certain conditions; Indicates the action The reward value, α represents the reward value based on the strategy. The output actions of each distributed power source are represented by R, which is the reward calculated by the reward formula based on the feedback from the power grid after the action α is executed.

7. The method of claim 2, wherein The parameters of the Actor policy network are updated by maximizing the policy entropy and the Q value, which is represented as: wherein is a loss function for the policy network, are network parameters for the policy network, [ ] denotes a weighted average over the state variable set . [ ] is an expected value of the action π selected by the policy , is a probability of the action selected by the policy π over the state variable set .

8. A power distribution network voltage model predictive control apparatus, characterized by, The method comprises the following steps: The power distribution network state acquisition module is configured to obtain real-time power grid states and predicted states of the power distribution network within an observation scale, wherein the observation scale is a time interval with a current time t0 as a starting point and a length d; the real-time power grid states and the predicted states comprise node voltages, distributed power outputs, load powers and line active powers; The control action generation module is configured to input the real-time power grid state and the predicted state into the pre-trained SAC-MPC agent, and output a control action in an action scale from a current time t0 as a starting point to a time interval with a length of l , and l ≤d; the control action is a reactive power output instruction assigned to the voltage control device. The instruction issuing module is configured to issue control actions to corresponding voltage control devices in the power distribution network, wherein only the control action of the next time step is issued to the voltage control devices; A feedback module is configured to acquire the power grid state after the voltage control device performs the control action and causes the state of the power distribution network environment to change, and feed back to the SAC-MPC agent for subsequent control.

9. An electronic device, comprising: The computer program comprises a computer program element for implementing the method according to any one of claims 1 to 7 when the computer program element is run by a processor.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one instruction, and the at least one instruction is executed by the processor to implement the method according to any one of claims 1 to 7.

Images