Active power distribution network power flow dynamic optimization method and system

Abstract

The invention provides an active power distribution network power flow dynamic optimization method and system, and the method comprises the steps: building a power distribution network digital model based on a pre-obtained optimization target and a pre-obtained power flow constraint; constructing a power distribution network power flow dynamic optimization decision process based on the power distribution network digital model to obtain a decision architecture model; based on a preset decision algorithm and the decision architecture model, training a pre-constructed initial agent until the initial agent meets a preset decision requirement, and obtaining a target agent; and carrying out power flow dynamic optimization on the power distribution network through the target intelligent agent based on power distribution network power flow data input in real time. The method solves the problem of poor reliability of power flow dynamic optimization of the power distribution network caused by falling into a local optimal solution when facing complex nonlinear constraints in the prior art. According to the method, the optimal decision is made for the real-time changing power flow data of the power distribution network, the continuous action space is effectively processed, local optimum is avoided, and the reliability of power flow dynamic optimization of the power distribution network is improved.

Description

Technical Field

[0001] This invention relates to the field of power distribution network optimization technology, and in particular to a method and system for dynamic power flow optimization in active power distribution networks. Background Technology

[0002] With the large-scale integration of distributed power sources into the distribution network, the distribution network has transformed from a traditional passive network into an active distribution network with active control capabilities. In the operation of the active distribution network, how to improve the reliability of dynamic optimization of power flow distribution has become a technical problem to be solved.

[0003] Currently, existing technologies for dynamic optimization of power flow distribution mainly employ numerical optimization methods such as Lagrange relaxation, augmented Lagrange relaxation, and alternating direction multiplier method. However, when dealing with complex nonlinear constraints and integer variables, these methods typically require complex convex relaxation or linearization processes, which may lead to decreased solution accuracy or the appearance of local optima. This is especially true as the scale of active distribution networks continues to expand and uncertainties increase, making it easier to increase computational complexity and failing to meet the requirements of dynamic power flow optimization. Furthermore, existing technologies also employ intelligent optimization algorithms such as particle swarm optimization, harmony search, and the cuckoo algorithm. While these methods can handle complex constraints, they are essentially heuristic methods based on stochastic search, and have inherent drawbacks such as susceptibility to local optima, reliance on experience for parameter adjustment, slow convergence speed, and unsatisfactory optimization efficiency and stability when facing high-dimensional continuous decision spaces. Therefore, it is evident that existing technologies are prone to getting trapped in local optima when facing complex nonlinear constraints, resulting in poor reliability in dynamic optimization of power flow in distribution networks. Summary of the Invention

[0004] To address the aforementioned issues, this invention proposes an active power flow dynamic optimization method and system for distribution networks. This method enables optimal decision-making based on real-time changing power flow data of the distribution network, effectively handles continuous action space to avoid getting trapped in local optima, and improves the reliability of power flow dynamic optimization in distribution networks.

[0005] To achieve the above objectives, embodiments of the present invention provide a method for dynamic optimization of power flow in an active distribution network, comprising: constructing a digital model of the distribution network based on pre-acquired optimization objectives and pre-acquired power flow constraints; constructing a dynamic optimization decision process for power flow in the distribution network based on the digital model of the distribution network to obtain a decision architecture model; training a pre-constructed initial agent based on a preset decision algorithm and the decision architecture model until the initial agent meets preset decision requirements to obtain a target agent; and performing dynamic optimization of the power flow in the distribution network through the target agent based on real-time input power flow data of the distribution network.

[0006] This invention proposes a dynamic power flow optimization method for active distribution networks. It constructs a digital model and decision architecture model of the distribution network, transforming the complex dynamic power flow optimization problem into a decision-making process that can be handled by reinforcement learning. This effectively solves the problem that traditional optimization methods are prone to getting trapped in local optima when dealing with high-dimensional nonlinear constraints. Then, deep reinforcement learning is used to train the agent online, generating a target agent that can adaptively track real-time changes in the distribution network's state and find the optimal solution from a global perspective. Thus, it enables optimal decision-making based on real-time changing power flow data of the distribution network, effectively handling the continuous action space to avoid getting trapped in local optima and improving the reliability of dynamic power flow optimization in distribution networks.

[0007] Furthermore, based on the pre-acquired optimization objectives and pre-acquired power flow constraints, a digital model of the distribution network is constructed, including: acquiring the output vector of the distribution network within a preset scheduling period; constructing the total active power loss based on the output vector, and using the total active power loss as the optimization objective; acquiring the topology of the distribution network, and constructing power flow constraints based on the topology; and constructing the digital model of the distribution network based on the optimization objectives and power flow constraints.

[0008] In the above scheme, the total active power loss of the system is used as the optimization objective. Power flow constraints are then constructed based on the distribution network topology. By incorporating the output vector and topological relationship into the mathematical model, the physical feasibility and security of the optimization process are ensured. Furthermore, it enables coordinated control of distributed power sources and energy storage systems, effectively reducing network losses and improving energy utilization efficiency. Therefore, by constructing a digital model of the distribution network, the operating characteristics of the distribution network are accurately described, providing a reliable data foundation for subsequent reinforcement learning training and contributing to improving the reliability of dynamic power flow optimization in the distribution network.

[0009] Furthermore, the topology of the distribution network is obtained, and power flow constraints are constructed based on the topology, including: obtaining the node connection relationship of the distribution network based on the topology; calculating the node voltage, active power and reactive power of each node based on the node connection relationship; and constructing power flow constraints based on the node voltage, active power between nodes and reactive power between nodes.

[0010] In the above scheme, the node connection relationships are obtained from the distribution network topology, and a power flow constraint system of node voltage, active power, and reactive power is constructed. This comprehensively considers the electrical characteristics and topological constraints of the distribution network, avoiding operational risks caused by voltage exceeding limits or power imbalance. This provides a clear state space and action boundaries for subsequent agent training and learning, helping to improve the reliability of dynamic power flow optimization in the distribution network.

[0011] Furthermore, a dynamic optimization decision-making process for power flow in the distribution network is constructed based on the digital model of the distribution network, resulting in a decision architecture model. This process includes: obtaining the active and reactive power demands of each node and constructing the dynamic optimization decision state of the power flow; constructing dynamic optimization decision actions based on the power flow constraints of the digital model of the distribution network; constructing dynamic optimization decision rewards based on the optimization objectives of the digital model of the distribution network; and obtaining the decision architecture model based on the dynamic optimization decision state, dynamic optimization decision actions, and dynamic optimization decision rewards of the power flow.

[0012] In the above scheme, the distribution network optimization problem is constructed as a Markov decision process that includes state, action and reward. The decision architecture model constructed in this way enables the agent to continuously learn the optimal strategy through interaction with the environment during the subsequent training process, which helps to improve the reliability of dynamic optimization of distribution network power flow.

[0013] Furthermore, based on a preset decision algorithm and decision architecture model, a pre-constructed initial agent is trained until it meets preset decision requirements to obtain a target agent. This includes: constructing a decision action network and a decision evaluation network based on the preset decision algorithm, the dynamic optimization decision state of the distribution network power flow, and the dynamic optimization decision actions of the distribution network power flow. Specifically, this includes: constructing a first decision action network loss function based on the preset decision algorithm and the dynamic optimization decision state and actions of the distribution network power flow, thus obtaining the decision action network; constructing a first target evaluation value and a first decision evaluation network loss function based on the preset decision algorithm and the dynamic optimization decision state and actions of the distribution network power flow, thus obtaining the decision evaluation network; and training the pre-constructed initial agent based on the decision action network and the decision evaluation network. This includes: interacting with the pre-constructed initial agent based on the optimization objective and power flow constraints to generate several... The process involves: running data and storing it in a preset experience pool; executing the agent training process: selecting running data from the preset experience pool based on preset sampling requirements, updating the decision action network and decision evaluation network, and calculating the second decision action network loss function, the second target evaluation value, and the second decision evaluation network loss function; training the initial agent based on the second decision action network loss function, the second target evaluation value, and the second decision evaluation network loss function, and ending the agent training process; iteratively updating the decision action network and decision evaluation network until the initial agent meets the preset decision requirements, thus obtaining the target agent, including: obtaining the second target evaluation value each time an agent training process is completed; if the second target evaluation value does not meet the preset decision requirements, re-executing the agent training process to iteratively update the initial agent; if the second target evaluation value output by the initial agent meets the preset decision requirements, then outputting the initial agent as the target agent.

[0014] In the above scheme, a decision-making action network and a decision evaluation network are constructed to handle policy generation and value assessment respectively, which significantly improves learning efficiency and stability. Then, an iterative update training mechanism and an experience playback mechanism are adopted to enable the agent to have strong generalization ability and real-time decision-making ability through end-to-end training, which helps to improve the reliability of power flow dynamic optimization in distribution networks.

[0015] Furthermore, based on real-time input distribution network power flow data, the distribution network is dynamically optimized through a target intelligent agent, including: receiving distribution network power flow data in real time and obtaining several distribution network states; inputting the distribution network states to the target intelligent agent and outputting several candidate actions corresponding to the several distribution network states; calculating the evaluation values ​​of the several candidate actions through the target intelligent agent, selecting the candidate actions whose evaluation values ​​meet the preset optimization requirements, and obtaining the target action; and performing dynamic power flow optimization on the distribution network based on the target action.

[0016] In the above scheme, the trained target agent is applied to a real power distribution network environment. By collecting system status in real time and outputting the optimal action, it can effectively track the rapid fluctuations of load and distributed power sources, meet the real-time optimization requirements of dynamic power flow, evaluate the evaluation value of candidate actions and select the optimal strategy to achieve closed-loop dynamic optimization control, and make optimal decisions on the real-time changing power flow data of the power distribution network. It effectively handles the continuous action space to avoid getting trapped in local optima and improves the reliability of dynamic optimization of power flow in the power distribution network.

[0017] This invention also provides an active distribution network power flow dynamic optimization system, comprising: a first model building module, a second model building module, a target agent acquisition module, and a power flow dynamic optimization module; the first model building module is used to build a distribution network digital model based on pre-acquired optimization objectives and pre-acquired power flow constraints; the second model building module is used to build a power flow dynamic optimization decision process for the distribution network based on the distribution network digital model, thereby obtaining a decision architecture model; the target agent acquisition module is used to train a pre-built initial agent based on a preset decision algorithm and the decision architecture model until the initial agent meets preset decision requirements, thereby obtaining a target agent; the power flow dynamic optimization module is used to perform power flow dynamic optimization of the distribution network through the target agent based on real-time input distribution network power flow data.

[0018] This invention proposes an active distribution network power flow dynamic optimization system. It constructs a digital model and decision architecture model of the distribution network, transforming the complex dynamic power flow optimization problem into a reinforcement learning-processable decision-making process. This effectively solves the problem of traditional optimization methods easily getting trapped in local optima when dealing with high-dimensional nonlinear constraints. Then, deep reinforcement learning is used to train the agent online, generating a target agent that can adaptively track real-time changes in the distribution network state and find the optimal solution from a global perspective. Thus, it enables optimal decision-making based on real-time changing power flow data of the distribution network, effectively handling continuous action space to avoid getting trapped in local optima and improving the reliability of dynamic power flow optimization in the distribution network. Attached Figure Description

[0019] Figure 1 A flowchart illustrating the steps of an active power distribution network dynamic optimization method according to a certain embodiment of the present invention; Figure 2 A schematic diagram of the target agent training process for an active power distribution network dynamic optimization method according to a certain embodiment of the present invention; Figure 3 This is a schematic diagram of the module structure of an active power distribution network dynamic optimization system provided in a certain embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1 See Figure 1 , Figure 1 This is a flowchart illustrating the steps of a dynamic power flow optimization method for an active distribution network according to a certain embodiment of the present invention. Figure 1 As shown in the figure, this embodiment of the invention proposes a dynamic power flow optimization method for active distribution networks, including steps 101 to 104, each step of which is as follows: Step 101: Based on the pre-acquired optimization objectives and pre-acquired power flow constraints, construct a digital model of the distribution network; Step 102: Construct a dynamic optimization decision-making process for power flow in the distribution network based on the digital model of the distribution network, and obtain the decision architecture model; Step 103: Based on the preset decision algorithm and decision architecture model, train the pre-constructed initial agent until the initial agent meets the preset decision requirements to obtain the target agent; Step 104: Based on the real-time input power flow data of the distribution network, the power flow of the distribution network is dynamically optimized by the target intelligent agent.

[0022] One possible implementation involves first defining the optimization objective and power flow constraints, and constructing an accurate digital model of the active distribution network. In this embodiment, minimizing the total active power loss of the system over the entire scheduling cycle (e.g., 24 hours) is typically the core optimization objective. Power flow constraints include: conventional branch power flow constraints, distributed generation node constraints, and energy storage system node constraints. Then, the accurate digital model of the active distribution network is converted into a decision-making process that can be solved by a deep reinforcement learning agent. In this embodiment, a Markov decision process is used as an example. Next, an initial agent is trained using a preset decision algorithm until it meets the preset decision requirements, resulting in a target agent. In this embodiment, the preset decision algorithm is explained using the Deep Deterministic Policy Gradient (DDPG) algorithm. Finally, the target agent is deployed in a real interactive environment to collect the state of the distribution network in real time. The target agent outputs the corresponding optimal strategy, and the power flow is dynamically optimized based on the optimal strategy.

[0023] This invention proposes a dynamic power flow optimization method for active distribution networks. It constructs a digital model and decision architecture model of the distribution network, transforming the complex dynamic power flow optimization problem into a decision-making process that can be handled by reinforcement learning. This effectively solves the problem that traditional optimization methods are prone to getting trapped in local optima when dealing with high-dimensional nonlinear constraints. Then, deep reinforcement learning is used to train the agent online, generating a target agent that can adaptively track real-time changes in the distribution network's state and find the optimal solution from a global perspective. Thus, it enables optimal decision-making based on real-time changing power flow data of the distribution network, effectively handling the continuous action space to avoid getting trapped in local optima and improving the reliability of dynamic power flow optimization in distribution networks.

[0024] A preferred embodiment involves constructing a digital model of a distribution network based on pre-acquired optimization objectives and pre-acquired power flow constraints, including: acquiring the output vector of the distribution network within a preset scheduling period; constructing the total active power loss based on the output vector and using the total active power loss as the optimization objective; acquiring the topology of the distribution network and constructing power flow constraints based on the topology; and constructing a digital model of the distribution network based on the optimization objective and power flow constraints.

[0025] One preferred implementation method uses minimizing the total active power loss of the system over the entire scheduling cycle (e.g., 24 hours) as the core optimization objective. This is achieved by obtaining the power output vector of the distribution network over the entire scheduling cycle, where the preset scheduling cycle is represented as 24 hours. The power output vector includes: the active power output vector of all energy storage systems, the reactive power output vector of all energy storage systems, and the reactive power output vector of all distributed photovoltaic units. The total active power loss is then constructed based on the power output vector of the distribution network, as expressed below: ; In the formula, F is the optimization objective, representing the total active power loss of the distribution network during the dispatch cycle. In this embodiment, the total active power loss is also related to the energy storage charging and discharging status and the photovoltaic active power output limit. As a branch in the distribution network, from node Pointing to node ; It is a set of branches, containing indices of all branches in the network; branch road The resistance; For flow through branch road The square of the current amplitude; This represents the active power output vector for all energy storage systems. This represents the reactive power output vector for all energy storage systems. This represents the reactive power output vector for all distributed photovoltaic units.

[0026] Then, based on the distribution network topology, the node connection relationships are obtained; based on these relationships, the node voltages, active power, and reactive power of each node are calculated; based on the node voltages, active power between nodes, and reactive power between nodes, power flow constraints are constructed; finally, based on the optimization objective and power flow constraints, a digital model of the distribution network is constructed, expressed as follows: ; In the above scheme, the total active power loss of the system is used as the optimization objective. Power flow constraints are then constructed based on the distribution network topology. By incorporating the output vector and topological relationship into the mathematical model, the physical feasibility and security of the optimization process are ensured. Furthermore, it enables coordinated control of distributed power sources and energy storage systems, effectively reducing network losses and improving energy utilization efficiency. Therefore, by constructing a digital model of the distribution network, the operating characteristics of the distribution network are accurately described, providing a reliable data foundation for subsequent reinforcement learning training and contributing to improving the reliability of dynamic power flow optimization in the distribution network.

[0027] A preferred embodiment involves obtaining the topology of a distribution network and constructing power flow constraints based on the topology, including: obtaining the node connection relationships of the distribution network based on the topology; calculating the node voltage, active power, and reactive power of each node based on the node connection relationships; and constructing power flow constraints based on the node voltage, the active power between nodes, and the reactive power between nodes.

[0028] In one preferred implementation, the power flow constraints include: conventional branch power flow constraints, distributed generation node constraints, and energy storage system node constraints. The conventional branch power flow constraints are used to ensure the safe and stable operation of the distribution network and need to satisfy corresponding physical laws and operational constraints. First, the connection relationships between distribution network nodes are obtained based on the distribution network topology. Then, the corresponding node voltages, node active power, and node reactive power are calculated based on these connections. In this embodiment, nodes i, j, and k with connection relationships are used for interpretation, where node k is the downstream neighbor of node j. Next, the voltage magnitudes of nodes i and j, the active power flowing from node j to node k, the active power flowing from node i to node j, the reactive power flowing from node j to node k, and the reactive power flowing from node i to node j are calculated. After obtaining the node voltages, the active power between nodes, and the reactive power between nodes, the physical law equations that the conventional branch power flow constraints need to satisfy are constructed. The specific expression is as follows: Node active power balance: ; Node reactive power balance: ; And the branch voltage drop equation: ; In the formula, The net active power injected into node j, ; The net reactive power injected into node j, ; Let be the active power flowing from node j to node k; k is the downstream neighboring node of node j. This indicates that k belongs to the set of downstream nodes of node j; Let be the active power flowing from node i to node j; Let be the reactive power flowing from node j to node k; Reactive power flowing from node i to node j; and Let be the voltage magnitudes at nodes i and j; The collection of all branches in a power distribution network. This indicates that the branch connects from node i to node j; It is a set of nodes, containing all nodes in the network; branch road The reactance; And the constraint equations are applied, with the following specific expressions: Relationship between branch current and power: ; and node voltage safety constraints: ; In the formula, This is the lower limit of the node's safe voltage. This refers to the upper limit of the node's safe voltage. Subsequently, to simplify control and enhance the model's universality, distributed power generation node constraints are applied to photovoltaic units using constant power factor control, expressed as follows: ; In the formula, For nodes The active power output of the photovoltaic unit; For nodes The reactive power output of the photovoltaic unit; For nodes The maximum active power output of the photovoltaic unit; The reactive power-active power ratio is a controlled variable. Finally, the energy storage system node constraints include: Energy storage capacity constraints: ; Charge and discharge power constraints: ; And the energy storage state evolution equation: ; In the formula, For at any time ,node The remaining power of the upper energy storage system; These are the lower and upper limits of the energy storage system's capacity. For at any time ,node The charging power of the energy storage system; For at any time ,node The discharge power of the energy storage system; For the system at time The total active power output and total active load of the distributed power sources; This represents the total number of energy storage nodes in the system. For the charging and discharging efficiency of energy storage systems; For time intervals.

[0029] In the above scheme, the node connection relationships are obtained from the distribution network topology, and a power flow constraint system of node voltage, active power, and reactive power is constructed. This comprehensively considers the electrical characteristics and topological constraints of the distribution network, avoiding operational risks caused by voltage exceeding limits or power imbalance. This provides a clear state space and action boundaries for subsequent agent training and learning, helping to improve the reliability of dynamic power flow optimization in the distribution network.

[0030] A preferred embodiment involves constructing a dynamic optimization decision-making process for power flow in a distribution network based on a digital model of the distribution network, resulting in a decision architecture model. This process includes: acquiring the active and reactive power demands of each node and constructing the dynamic optimization decision state of the power flow; constructing dynamic optimization decision actions based on the power flow constraints of the digital model of the distribution network; constructing dynamic optimization decision rewards based on the optimization objectives of the digital model of the distribution network; and obtaining the decision architecture model based on the dynamic optimization decision state, dynamic optimization decision actions, and dynamic optimization decision rewards of the power flow.

[0031] One preferred implementation involves transforming the dynamic optimal power flow problem into a Markov decision process solvable by a deep reinforcement learning agent, based on the constructed distribution network digital model. Specifically, this involves constructing the dynamic optimization decision state of the distribution network power flow based on the active and reactive power demands of each node, constructing the dynamic optimization decision actions of the distribution network power flow based on the power flow constraints of the distribution network digital model, and constructing the dynamic optimization decision reward of the distribution network power flow based on the optimization objective of the distribution network digital model. The Markov decision process includes constructing the dynamic optimization decision state of the distribution network power flow. Distribution network power flow dynamic optimization decision-making actions Distribution network power flow dynamic optimization decision reward The specific expression is as follows: Distribution network power flow dynamic optimization decision state : ; Distribution network power flow dynamic optimization decision-making actions : ; Distribution network power flow dynamic optimization decision reward : ; In the formula, The current active power loss; For nodes The active power output of the upper energy storage unit; For nodes The reactive power output of the upper energy storage unit; It is a positive penalty coefficient; For counting functions that violate constraints.

[0032] The constructed Markov decision process is thus used as a decision architecture model.

[0033] In the above scheme, the distribution network optimization problem is constructed as a Markov decision process that includes state, action and reward. The decision architecture model constructed in this way enables the agent to continuously learn the optimal strategy through interaction with the environment during the subsequent training process, which helps to improve the reliability of dynamic optimization of distribution network power flow.

[0034] A preferred embodiment involves training a pre-constructed initial agent based on a preset decision algorithm and decision architecture model until the initial agent meets preset decision requirements to obtain a target agent. This includes: constructing a decision action network and a decision evaluation network based on the preset decision algorithm, the dynamic optimization decision state of the distribution network power flow, and the dynamic optimization decision actions of the distribution network power flow; constructing a first decision action network loss function based on the preset decision algorithm and the dynamic optimization decision state and actions of the distribution network power flow, thus obtaining the decision action network; constructing a first target evaluation value and a first decision evaluation network loss function based on the preset decision algorithm and the dynamic optimization decision state and actions of the distribution network power flow, thus obtaining the decision evaluation network; and training the pre-constructed initial agent based on the decision action network and the decision evaluation network, wherein: interacting with the pre-constructed initial agent based on the optimization target and power flow constraints to generate several... The process involves: 1) Obtaining operational data and storing it in a preset experience pool; 2) Performing agent training: Selecting operational data from the preset experience pool based on preset sampling requirements, updating the decision action network and decision evaluation network, and calculating the second decision action network loss function, the second target evaluation value, and the second decision evaluation network loss function; 3) Training the initial agent based on the second decision action network loss function, the second target evaluation value, and the second decision evaluation network loss function, ending the agent training process; 4) Iteratively updating the decision action network and decision evaluation network until the initial agent meets the preset decision requirements, obtaining the target agent, including: obtaining the second target evaluation value each time an agent training process is completed; 5) If the second target evaluation value does not meet the preset decision requirements, re-executing the agent training process to iteratively update the initial agent; 6) If the second target evaluation value output by the initial agent meets the preset decision requirements, outputting the initial agent as the target agent.

[0035] One preferred implementation involves training an initial agent using a preset decision-making algorithm until the initial agent meets the preset decision requirements, thus obtaining a target agent. Specifically, the agent is designed and trained using the Deep Deterministic Policy Gradient (DDPG) algorithm. The core components of the DDPG algorithm include: a decision-action network, represented as an Actor network; a decision-evaluation network, represented as a Critic network; and a target Q-value. The expression is as follows: Critic network loss function: ; Target Q value calculation; ; Actor network loss function: ; Target network soft update: ; In the formula, Let be the loss function of the Critic network; Let be the loss function of the Actor network; Batch size; The target Q value; The Q-value estimated for the online Critic network; For a moment The actual reward received; Discount factor; The Q-value estimated for the target Critic network; The parameters for the target Critic network and the target Actor network; Parameters for the online Critic network and the online Actor network; This is the soft update coefficient.

[0036] During each training iteration, the loss functions of the Critic network, Actor network, and target Q-value at the current training moment are represented as the first decision evaluation network loss function, the first decision action network loss function, and the first target evaluation value. After training is completed at the current moment, the loss functions of the Critic network, Actor network, and target Q-value are represented as the second decision evaluation network loss function, the second decision action network loss function, and the second target evaluation value. By continuously iteratively updating the parameters of the Actor network and Critic network, the agent learns the optimal strategy that satisfies the optimization objective function and constraints. After training, the Actor network parameters are fixed, resulting in the target agent. For a specific explanation, see [link to explanation]. Figure 2 , Figure 2 This is a schematic diagram of the target agent training process for an active power distribution network dynamic optimization method according to a certain embodiment of the present invention; as shown below. Figure 2 As shown, before training the agent, the core optimization objective function and power flow constraints of the power system are first obtained. The reward function for reinforcement learning is then designed based on the optimization objective and power flow constraints. This guides the agent to learn the optimal strategy. During the agent's training process, the agent interacts with the power distribution network environment, and the experience tuples generated from each interaction are processed. The experience data is stored in a fixed-size experience replay buffer (experience pool). Based on this, small batches of randomly sampled historical experience data and randomly introduced Gaussian noise are input into the action network (Actor network, with parameters ) in the online network, according to the core optimization objective and the Deep Deterministic Policy Gradient (DDPG) algorithm. ) and evaluation network (Critic network, parameters are Collaborative learning and parameter updates are performed; the Actor network is responsible for adjusting parameters based on the system state. Output action or action distribution The Critic network is responsible for evaluating the value of state-action pairs. This process measures the quality of actions; through iterative updates based on experience playback, the agent eventually learns the optimal strategy that satisfies the objective function and constraints. At this point, the target action network (Actor network, with parameters...) is obtained. ) and the target evaluation network (Critic network, with parameters of ) and That is, the target intelligent agent is obtained. It is worth mentioning that the optimal strategy that satisfies the optimization objective function and constraints is represented by the preset decision requirements in this embodiment, and random sampling of small batches is represented by the preset sampling requirements.

[0037] In the above scheme, a decision-making action network and a decision evaluation network are constructed to handle policy generation and value assessment respectively, which significantly improves learning efficiency and stability. Then, an iterative update training mechanism and an experience playback mechanism are adopted to enable the agent to have strong generalization ability and real-time decision-making ability through end-to-end training, which helps to improve the reliability of power flow dynamic optimization in distribution networks.

[0038] A preferred embodiment involves dynamically optimizing the power flow of a distribution network based on real-time input power flow data, using a target intelligent agent. This includes: receiving power flow data in real-time and acquiring several distribution network states; inputting the distribution network states to the target intelligent agent and outputting several candidate actions corresponding to those states; calculating evaluation values ​​for the candidate actions using the target intelligent agent, selecting candidate actions whose evaluation values ​​meet preset optimization requirements to obtain the target action; and performing dynamic power flow optimization on the distribution network based on the target action.

[0039] One preferred implementation involves acquiring system status data from the environment in real time via a target intelligent agent. , including: nodes active demand ;node reactive power demand ;node Photovoltaic power output in, , The process involves creating a set of nodes and then outputting a policy based on the pre-trained decision algorithm in the target agent. Specifically, the Actor network first generates candidate actions and sets the state... The input is fed into a pre-trained action network (Actor network), and the Actor network outputs an action distribution or a set of candidate actions. Each candidate action includes: node Active power output of the upper energy storage unit ;node Reactive power output of the upper energy storage unit Other control variables, such as the setpoint of the photovoltaic inverter. Then, the target network evaluates and selects the optimal action, assigning each candidate action to a specific action. With state The inputs are combined and fed into the Critic network to calculate the value of each state-action pair. Then, select the action with the highest value as the final action. ,Right now: In this embodiment, the preset optimization requirement is represented by the action with the highest value; finally, the final action is... The data is then distributed to relevant distribution network nodes, such as photovoltaic inverters and energy storage systems, for execution to achieve dynamic optimal power flow control of the power grid. After execution, the distribution network state is updated, and the agent continues to collect the state at the next moment, repeating the above process to dynamically optimize the power flow of the distribution network.

[0040] In the above scheme, the trained target agent is applied to a real power distribution network environment. By collecting system status in real time and outputting the optimal action, it can effectively track the rapid fluctuations of load and distributed power sources, meet the real-time optimization requirements of dynamic power flow, evaluate the evaluation value of candidate actions and select the optimal strategy to achieve closed-loop dynamic optimization control, and make optimal decisions on the real-time changing power flow data of the power distribution network. It effectively handles the continuous action space to avoid getting trapped in local optima and improves the reliability of dynamic optimization of power flow in the power distribution network.

[0041] Example 2 See Figure 3 , Figure 3 This is a schematic diagram of the module structure of an active power distribution network dynamic optimization system according to a certain embodiment of the present invention. Figure 3 As shown in the figure, this embodiment of the invention also provides an active distribution network power flow dynamic optimization system, including: a first model construction module 201, a second model construction module 202, a target agent acquisition module 203, and a power flow dynamic optimization module 204; the first model construction module 201 is used to construct a distribution network digital model based on pre-acquired optimization objectives and pre-acquired power flow constraints; the second model construction module 202 is used to construct a power flow dynamic optimization decision process for the distribution network based on the distribution network digital model, and obtain a decision architecture model; the target agent acquisition module 203 is used to train a pre-constructed initial agent based on a preset decision algorithm and decision architecture model until the initial agent meets the preset decision requirements, and obtain a target agent; the power flow dynamic optimization module 204 is used to perform power flow dynamic optimization of the distribution network through the target agent based on real-time input distribution network power flow data.

[0042] This invention proposes an active distribution network power flow dynamic optimization system. It constructs a digital model and decision architecture model of the distribution network, transforming the complex dynamic power flow optimization problem into a reinforcement learning-processable decision-making process. This effectively solves the problem of traditional optimization methods easily getting trapped in local optima when dealing with high-dimensional nonlinear constraints. Then, deep reinforcement learning is used to train the agent online, generating a target agent that can adaptively track real-time changes in the distribution network state and find the optimal solution from a global perspective. Thus, it enables optimal decision-making based on real-time changing power flow data of the distribution network, effectively handling continuous action space to avoid getting trapped in local optima and improving the reliability of dynamic power flow optimization in the distribution network.

[0043] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

[0044] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the described specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0045] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

Claims

1. An active power distribution network flow dynamic optimization method, characterized in that, The method comprises the following steps: constructing a digital model of the power distribution network based on a pre-acquired optimization target and a pre-acquired power flow constraint; constructing a power flow dynamic optimization decision process of the power distribution network based on the digital model of the power distribution network, to obtain a decision architecture model; training a pre-constructed initial agent based on a preset decision algorithm and the decision architecture model until the initial agent meets a preset decision requirement, to obtain a target agent; performing power flow dynamic optimization on the power distribution network through the target agent based on real-time input power flow data of the power distribution network.

2. The active power distribution network dynamic power flow optimization method of claim 1, wherein, The step of constructing the digital model of the power distribution network based on the pre-acquired optimization target and the pre-acquired power flow constraint comprises the following steps: acquiring an output vector of the power distribution network in a preset scheduling period; constructing total active power loss based on the output vector, and taking the total active power loss as the optimization target; acquiring a topological structure of the power distribution network, and constructing the power flow constraint according to the topological structure; constructing the digital model of the power distribution network based on the optimization target and the power flow constraint.

3. The active power distribution network dynamic power flow optimization method of claim 2, wherein, The step of acquiring the topological structure of the power distribution network and constructing the power flow constraint according to the topological structure comprises the following steps: acquiring a node connection relationship of the power distribution network based on the topological structure of the power distribution network; calculating node voltage, active power of each node and reactive power of each node based on the node connection relationship of the power distribution network; constructing the power flow constraint based on the node voltage, active power between nodes and reactive power between nodes.

4. The active power distribution network dynamic power flow optimization method of claim 3, wherein, The step of constructing the power flow dynamic optimization decision process of the power distribution network based on the digital model of the power distribution network, to obtain the decision architecture model, comprises the following steps: acquiring active demand and reactive demand of each node, to construct a power flow dynamic optimization decision state of the power distribution network; constructing a power flow dynamic optimization decision action of the power distribution network based on the power flow constraint of the digital model of the power distribution network; constructing a power flow dynamic optimization decision reward of the power distribution network based on the optimization target of the digital model of the power distribution network; obtaining the decision architecture model based on the power flow dynamic optimization decision state of the power distribution network, the power flow dynamic optimization decision action of the power distribution network and the power flow dynamic optimization decision reward of the power distribution network.

5. The active power distribution network dynamic optimization method of claim 4, wherein, The step of training the pre-constructed initial agent based on the preset decision algorithm and the decision architecture model until the initial agent meets the preset decision requirement, to obtain the target agent, comprises the following steps: constructing a decision action network and a decision evaluation network based on the preset decision algorithm, the power flow dynamic optimization decision state of the power distribution network and the power flow dynamic optimization decision action of the power distribution network; training the pre-constructed initial agent based on the decision action network and the decision evaluation network; iteratively updating the decision action network and the decision evaluation network until the initial agent meets the preset decision requirement, to obtain the target agent.

6. The active power distribution network dynamic optimization method of claim 5, wherein, The step of constructing the decision action network and the decision evaluation network based on the preset decision algorithm, the power flow dynamic optimization decision state of the power distribution network and the power flow dynamic optimization decision action of the power distribution network comprises the following steps: constructing a first decision action network loss function according to the power flow dynamic optimization decision state of the power distribution network and the power flow dynamic optimization decision action of the power distribution network based on the preset decision algorithm, to obtain the decision action network; construct a first target evaluation value and a first decision evaluation network loss function based on the power distribution network dynamic optimization decision state and the power distribution network dynamic optimization decision action according to the preset decision algorithm, and obtain a decision evaluation network.

7. The active power distribution network dynamic power flow optimization method of claim 5, wherein, Based on the decision action network and the decision evaluation network, the pre-constructed initial agent is trained, including: Based on the optimization target and the power flow constraint, the pre-constructed initial agent is interacted to generate a plurality of running data and store them in a preset experience pool; The agent training process is executed: Based on the preset sampling requirement, the running data are selected from the preset experience pool, the decision action network and the decision evaluation network are updated, and a second decision action network loss function, a second target evaluation value and a second decision evaluation network loss function are calculated; Based on the second decision action network loss function, the second target evaluation value and the second decision evaluation network loss function, the initial agent is trained, and the agent training process is ended.

8. The active power distribution network dynamic power flow optimization method of claim 7, wherein, The decision action network and the decision evaluation network are iteratively updated until the initial agent meets the preset decision requirement, and a target agent is obtained, including: The second target evaluation value is obtained every time the agent training process is completed; If the second target evaluation value does not meet the preset decision requirement, the agent training process is re-executed to iteratively update the initial agent; If the second target evaluation value output by the initial agent meets the preset decision requirement, the initial agent is output as the target agent.

9. The active power distribution network dynamic power flow optimization method of any one of claims 1 to 8, wherein, Based on the real-time input power distribution network power flow data, the power distribution network is dynamically optimized by the target agent, including: The power distribution network power flow data are received in real time to obtain a plurality of power distribution network states; The power distribution network states are input into the target agent to output a plurality of candidate actions corresponding to the power distribution network states; The evaluation values of the plurality of candidate actions are calculated by the target agent, a candidate action with an evaluation value meeting a preset optimization requirement is selected, and a target action is obtained; Based on the target action, the power distribution network is dynamically optimized.

10. An active power distribution network flow dynamic optimization system, characterized by, An active power distribution network dynamic optimization method is executed, including: A first model construction module, a second model construction module, a target agent acquisition module and a power flow dynamic optimization module; The first model construction module is used to construct a power distribution network digital model based on a pre-acquired optimization target and a pre-acquired power flow constraint; The second model construction module is used to construct a power distribution network dynamic optimization decision process based on the power distribution network digital model to obtain a decision architecture model; The target agent acquisition module is used to train a pre-constructed initial agent until the initial agent meets a preset decision requirement to obtain a target agent based on a preset decision algorithm and the decision architecture model; The power flow dynamic optimization module is used to dynamically optimize the power distribution network by the target agent based on real-time input power distribution network power flow data.

Images