Main network load transfer method, system and device based on multi-agent collaborative optimization and medium

Abstract

The invention discloses a main network load transfer method, system, equipment and medium based on multi-agent collaborative optimization, and the method comprises the steps: dividing a power distribution network into a plurality of feeder groups, deploying agents in each feeder group, and constructing a multi-agent collaborative architecture; modeling a multi-agent voltage stability problem into a distributed partially observable Markov decision process, and optimizing the running state of the power distribution network in a voltage safety area through a multi-target reward mechanism fusing voltage safety constraints; under a centralized training and decentralized execution framework, a cooperative dominant function is introduced, contribution of actions of each intelligent agent and an interconnection switch and a section switch controlled by the intelligent agent to overall performance is quantified, and a cooperative control strategy is formed; and dynamically adjusting the states of a multi-agent interconnection switch and a section switch based on a cooperative control strategy to realize multi-level load transfer. The method can adapt to the operation requirements of a complex power grid with high-proportion new energy access and severe net load fluctuation, and improves the active risk prevention and control capability, the operation economy and the power supply reliability.

Description

Technical Field

[0001] This invention relates to the technical field of distribution network self-healing, and in particular to a main grid load transfer method, system, equipment and medium based on multi-agent collaborative optimization. Background Technology

[0002] The core drawbacks of existing centralized master-station driven self-healing technology are: extremely high communication dependence; the master station needs to maintain real-time data interaction with all terminals, and once communication is interrupted, the entire self-healing process immediately halts, extending fault handling time from minutes to hours; master station system failure can paralyze the entire network's self-healing capability, failing to cope with the combined risks of master station failure and line faults; in large-scale power grids, centralized computing requires traversing massive switch combinations, with power flow calculations taking over 10 seconds, making it difficult to meet the requirement of second-level recovery after a fault. The core drawbacks of distributed terminal autonomous self-healing technology are: local optimization leading to global suboptimal results; terminals making decisions based solely on local data may lead to local recovery but global overload. Faced with complex faults spanning feeders and transformers, distributed terminals cannot form a global collaborative strategy, requiring manual intervention for coordination. Information conflicts are prone to occur when multiple terminals communicate in parallel, such as two terminals simultaneously vying for control of the tie switch, causing repeated switch operations.

[0003] The core drawbacks of existing deterministic distribution network reconfiguration technologies are: inability to handle uncertainty, neglect of load fluctuations and the randomness of distributed generation (DG) output; reconfiguration schemes generated based on predicted values ​​often result in voltage exceeding limits and line overloads in actual operation; traditional genetic algorithms have slow convergence speed in complex power grids and are prone to getting stuck in local optima due to improper selection-crossing operations, resulting in a network loss reduction rate of less than 50% of the theoretical value. Distortion of measurement data can directly lead to the failure of reconfiguration schemes and even trigger pseudo-reconfigurations. The core drawbacks of traditional feeder automation (FA) technologies are: local reclosing FAs require step-by-step reclosing of switches according to a fixed sequence, resulting in permanent fault handling time exceeding 30 seconds, and power outage time in non-faulty sections being more than 5 times that of modern technologies; local FAs cannot monitor the load on the opposite line in real time during power transfer, potentially transferring heavy-load feeder loads to light-load feeders, leading to overload tripping at the receiving end; the power transfer time limit of tie switches is a fixed value preset manually, without adjustment based on peak-valley load differences, making it prone to cascading faults during peak hours due to untimely power transfer. Summary of the Invention

[0004] In view of the aforementioned existing problems, this invention is proposed. Therefore, this invention provides a mains load transfer method, system, device, and medium based on multi-agent cooperative optimization to solve the problems mentioned in the background art.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, embodiments of the present invention provide a main grid load transfer method based on multi-agent cooperative optimization, comprising: dividing the distribution network into several feeder groups, and deploying agents in each feeder group to construct a multi-agent cooperative architecture; The multi-agent voltage stability problem is modeled as a distributed partially observable Markov decision process. By integrating a multi-objective reward mechanism with voltage safety constraints, the operation state of the distribution network is optimized within the voltage safety region. Under the framework of centralized training and distributed execution, a cooperative advantage function is introduced to quantify the contribution of each agent and its controlled contact switches and segmented switches to the global performance, thereby forming a cooperative control strategy. Based on the aforementioned collaborative control strategy, the states of multi-agent interconnection switches and segmented switches are dynamically adjusted to achieve multi-level load transfer.

[0006] As a preferred embodiment of the main grid load transfer method based on multi-agent cooperative optimization described in this invention, the step of modeling the multi-agent voltage stability problem as a distributed partially observable Markov decision process includes: defining the state space, observation space, joint action space, state transition function, shared reward function, and discount factor. The state space is composed of environmental state and action-related state. The environmental state includes the active power injection and reactive power injection of each renewable energy node at the current time step, as well as the active power demand and reactive power demand of each load node. The action-related state includes the active power, reactive power, reactive power compensation value, transformer tap position, and node voltage amplitude flowing into each node. Each agent can only observe local state information related to its electrical location, including the voltage amplitude, power injection, load demand, and renewable energy output of its node. The combined action space output is a normalized reactive power adjustment command, used to control the reactive power compensation device; the actions of all agents are spliced ​​together to form a combined action and act on the environment; The shared reward function is the negative sum of voltage over-limit penalty, network loss penalty, and operating cost penalty.

[0007] As a preferred embodiment of the main grid load transfer method based on multi-agent cooperative optimization described in this invention, the optimization of the distribution network operation status within the voltage safety area by integrating a multi-objective reward mechanism with voltage safety constraints includes: constructing a fully cooperative multi-agent reinforcement learning system, wherein each agent generates reactive power adjustment actions based on local observations and shares a unified reward function; Before executing an action, the execution network performs a voltage safety check on the current system state. If the voltage of any node exceeds the voltage safety threshold, the current training or control process is immediately terminated. Based on the degree of deviation of the voltage of each node from the voltage safety threshold, the operating state is divided into three regions. When the voltage is not greater than the first voltage threshold and not less than the second voltage threshold, the current operating range is determined to be the first safe zone. When the voltage is less than the second voltage threshold and not less than the third voltage threshold, or the voltage is not greater than the fourth voltage threshold and greater than the first voltage threshold, the current operating range is determined to be the second safe zone. When the voltage is less than the third voltage threshold or greater than the fourth voltage threshold, the current operating range is determined to be a danger zone.

[0008] As a preferred embodiment of the main network load transfer method based on multi-agent cooperative optimization described in this invention, it further includes: When the system is in the first safe zone, the reward function primarily minimizes the operating cost; when the system is in the second safe zone, the reward function primarily minimizes the voltage deviation. The expected cumulative reward function under hard constraints is defined as follows: in, For strategy The expected cumulative reward is as follows: Let the state value loss function be... For the KL divergence term, Let be the joint action consisting of the actions of all agents at time t.

[0009] Each optimal joint strategy satisfy: As a preferred embodiment of the main network load transfer method based on multi-agent cooperative optimization described in this invention, the cooperative advantage function introduced under the centralized training and distributed execution framework includes: The cooperative advantage function is expressed as follows: in, To remove intelligent agents The actions of all other intelligent agents, For intelligent agents The local state value function.

[0010] As a preferred embodiment of the main network load transfer method based on multi-agent cooperative optimization described in this invention, the following is included: quantifying the contribution of each agent and its controlled interconnection switches and segment switches to the global performance to form a cooperative control strategy: Define a cooperation metric for agents to evaluate their contribution to the overall system performance. The cooperation metric is expressed as: in, Indicates the presence of intelligent agents Total reward during the action, Indicates the exclusion of intelligent agents Total reward during the action, The larger the value, the stronger the agent. The higher the contribution to the overall system performance; The actions of the physical devices controlled by the intelligent agent are associated with its policy output. The physical devices include feeder tie switches, transformer tie switches, substation tie switches, and sectionalizing switches. By analyzing the changes in system operating indicators before and after the switch action, the contribution value of the switch to the photovoltaic absorption capacity and load power supply capacity in a specific scenario is quantified, and the contribution value is mapped to the corresponding intelligent agent's collaborative metric. Construct an objective function that integrates collaborative metrics to guide agent policy updates: in, This is an adjustment coefficient used to control the weight of the collaboration metric in the objective function; During the intensive training phase, each agent can acquire global state information and use the objective function as the policy gradient objective to update its agent network parameters. When the cooperation metric meets a preset threshold, the policy tends to select the cooperative action that is most beneficial to the system as a whole. The critic network updates its parameters by minimizing the Q-value prediction error, and its joint loss function is: in, b This indicates the batch size, representing the sampling of multiple trajectories from the experience replay buffer for synchronous training; Iteratively evaluate and improve the execution strategy until convergence is achieved to obtain the optimal collaborative control strategy.

[0011] As a preferred embodiment of the main grid load transfer method based on multi-agent cooperative optimization described in this invention, the method for dynamically adjusting the states of multi-agent interconnection switches and segmented switches based on the cooperative control strategy to achieve multi-level load transfer includes: establishing a multi-period reconfiguration model, wherein the multi-period reconfiguration model includes power flow constraints, security constraints, radial network topology constraints, photovoltaic power generation constraints, and load shedding constraints. By utilizing a multi-agent system to coordinate the interactions between different levels, when an overload or voltage limit is exceeded in any area, a corresponding switching action is triggered. Lower-level resources are prioritized for local adjustments. If the problem cannot be solved, higher-level resources are activated to participate in regulation. Based on the quantitative analysis results of the switch contribution, a long-term optimization strategy is formulated, and the action rules of each agent and its control device are regularly evaluated and adjusted.

[0012] Secondly, the present invention provides a main grid load transfer system based on multi-agent collaborative optimization, comprising: an agent deployment module, used to divide the distribution network into several feeder groups and deploy agents in each feeder group to construct a multi-agent collaborative architecture; The multi-objective reward modeling module is used to model the multi-agent voltage stability problem as a distributed partially observable Markov decision process. By integrating the multi-objective reward mechanism with voltage safety constraints, the operation state of the distribution network is optimized within the voltage safety region. The strategy generation module is used to introduce a cooperative advantage function under the centralized training and distributed execution framework, quantify the contribution of each agent and its controlled contact switches and segmented switches to the global performance, and form a cooperative control strategy. The multi-level load transfer execution module is used to dynamically adjust the states of the multi-agent interconnection switch and segment switch based on the aforementioned collaborative control strategy, thereby realizing multi-level load transfer.

[0013] Thirdly, the present invention provides an electronic device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the main network load transfer method based on multi-agent cooperative optimization.

[0014] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the main network load transfer method based on multi-agent cooperative optimization.

[0015] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention constructs a distributed partially observable Markov model, which combines a reward function with triple penalties for voltage overruns, network losses, and operating costs. This allows for early prediction of voltage overruns and line overload risks caused by fluctuations in renewable energy output and load changes. At the same time, the voltage is strictly controlled within a safe range through a hard constraint function. When the voltage exceeds the critical range, training is terminated, avoiding false reconstructions or cascading failures. Compared with deterministic reconstruction techniques, which are sensitive to measurement anomalies and cannot cope with uncertainties, this invention significantly improves fault tolerance and risk prediction capabilities.

[0016] The system features an innovative three-tier architecture of local autonomy, regional collaboration, and global coordination. It enables real-time and periodic interaction of data between intelligent agents in feeders, transformers, and substations. By quantifying the collaborative and cooperative advantage functions through a joint Q function, it accurately measures the contribution of each intelligent agent to the system. This approach avoids the problems of missing cross-regional collaboration and information conflicts in distributed terminals, eliminates the need for a centralized master station to traverse massive switch combinations, reduces power flow control time, and avoids the risk of network-wide self-healing paralysis caused by a single point of failure in the master station.

[0017] By adopting a decoupled architecture for feeder groups and a multi-period load transfer design, low-level reconfiguration is prioritized according to the feeder, transformer, and substation levels to reduce the dimensionality of complex network solutions. By combining the Monte Carlo method to generate typical scenario samples, the transfer strategy for different time periods is dynamically optimized, which can flexibly adapt to the randomness of new energy output and net load fluctuations, and significantly improve the photovoltaic absorption capacity.

[0018] The system achieves dual optimization of operational economy and efficiency. On one hand, the network loss penalty term guides the agent to reduce wire and transformer losses, while the normalized action's operating cost penalty term avoids wasting reactive power resources. Compared to traditional technologies, which have insufficient network loss reduction rates, this approach gets closer to the optimal network loss level. On the other hand, hierarchical reconstruction breaks down the global problem into local sub-regions, reducing the number of controllable switches in a single optimization. This significantly improves the solution speed compared to global reconstruction, increases sample collection efficiency, and avoids wasting computational resources.

[0019] In fault scenarios, by first implementing self-governance within a group and then coordinating with other groups, the non-faulty sections can quickly transfer loads, addressing the issue of long power outage times in non-faulty sections and improving power restoration efficiency. At the same time, the multi-level power transfer capability across feeders, transformers, and substations, combined with hierarchical protection and self-healing linkage, effectively reduces power outages caused by untimely power transfer during peak hours, achieving rapid power restoration during faults, fewer power outages during peak hours, and high power supply reliability. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a schematic diagram of a main network load transfer method based on multi-agent cooperative optimization according to an embodiment of the present invention; Figure 2 This is a topology diagram of a main network load transfer method based on multi-agent cooperative optimization according to an embodiment of the present invention; Figure 3 This is a result diagram of a main network load transfer method based on multi-agent cooperative optimization according to an embodiment of the present invention. Detailed Implementation

[0021] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0022] Example 1, referring to Figures 1-3 As one embodiment of the present invention, this embodiment provides a main network load transfer method based on multi-agent cooperative optimization, such as... Figure 1 As shown, it includes: S100: Divide the distribution network into several feeder groups, and deploy intelligent agents in each feeder group to build a multi-agent collaborative architecture; S200: The multi-agent voltage stability problem is modeled as a distributed partially observable Markov decision process. By integrating a multi-objective reward mechanism that incorporates voltage safety constraints, the operation status of the distribution network is optimized within the voltage safety region. S300: Under the framework of centralized training and distributed execution, a cooperative advantage function is introduced to quantify the contribution of each agent and its controlled contact switches and segmented switches to the global performance, thereby forming a cooperative control strategy. S400: Based on the aforementioned collaborative control strategy, dynamically adjust the states of the multi-agent interconnection switch and segmented switch to achieve multi-level load transfer.

[0023] It should be noted that existing technologies, such as centralized master station-driven self-healing technology, suffer from several drawbacks. These include a strong reliance on communication and the risk of single-point failures at the master station, which can lead to stagnation in the self-healing process and insufficient real-time performance. Distributed terminal autonomous technology, which relies solely on local data for decision-making, results in local optimization but lacks global suboptimal performance and cross-regional collaborative capabilities. Deterministic distribution network reconfiguration technology cannot cope with net load fluctuations and the randomness of renewable energy output, making the scheme prone to failure. Traditional feeder automation technology has long fault handling times and poor power transfer security. Static topology grid zoning technology cannot balance real-time equipment risks due to the rigidity of zoning. Furthermore, there are other comprehensive technical defects, such as inaccurate identification of multi-level reconfiguration needs of the main and distribution networks and the disconnect between hierarchical intelligent protection and self-healing actions, leading to low power supply reliability. This invention aims to address the technical challenges of constructing a proactive risk prevention and control mechanism to anticipate multi-dimensional operational risks, designing a multi-level collaborative architecture of feeders, transformers, and substations to achieve efficient global coordination, adapting to net load timing characteristics to optimize time-of-use load transfer strategies, accurately identifying the multi-level reconfiguration needs of the main and distribution networks, and establishing a hierarchical protection and self-healing linkage mechanism to shorten fault response time. This will enable the grid hierarchical self-healing technology to adapt to the complex grid operation needs of high-proportion renewable energy access and drastic net load fluctuations, thereby improving proactive risk prevention and control capabilities, operational economy, and power supply reliability.

[0024] In this embodiment of the invention, the step S200 of modeling the multi-agent voltage stability problem as a distributed partially observable Markov decision process includes: defining the state space, observation space, joint action space, state transition function, shared reward function, and discount factor; The state space is composed of environmental state and action-related state. The environmental state includes the active power injection and reactive power injection of each renewable energy node at the current time step, as well as the active power demand and reactive power demand of each load node. The action-related state includes the active power, reactive power, reactive power compensation value, transformer tap position, and node voltage amplitude flowing into each node. Each agent can only observe local state information related to its electrical location, including the voltage amplitude, power injection, load demand, and renewable energy output of its node. The combined action space output is a normalized reactive power adjustment command, used to control the reactive power compensation device; the actions of all agents are spliced ​​together to form a combined action and act on the environment; The shared reward function is the negative sum of voltage over-limit penalty, network loss penalty, and operating cost penalty.

[0025] Furthermore, the multi-agent voltage stability problem is described as a distributed partially observable Markov decision process, which is set as a fully cooperative multi-agent reinforcement learning task and defined by tuples. .in It represents the state space of a multi-agent model, which includes the global state of a novel distributed network; and These represent the observation space and the action space, respectively. The observation information includes the state values ​​and additional state information of all agents. P This is the state transition function. This is the discount factor. The agent obtains current observations from the training environment of the power distribution network simulation and outputs joint actions to the environment. At time... t The combined action, consisting of the actions of all agents, serves as the environmental input, denoted as... In this fully cooperative multi-agent model, all agents share a unified reward function. r At each time step t Each intelligent agent m Observe its local state Based on this observation, the agent generates reactive power regulation actions through its execution network. .

[0026] Specifically, the state space mainly consists of two parts: the environment state and the action-related state. The environmental state is represented as follows: in, Indicates the time of renewable energy nodes Active power injection, Indicates the time of renewable energy nodes reactive power injection, Indicates the load over time The active power demand, Indicates the load over time The reactive power demand.

[0027] Action-related states are represented as follows: in, Indicates the inflow node active power, Indicates the inflow node reactive power, Indicates time The reactive power compensation value, Indicates the tap position of the switch. Represents a node The voltage amplitude.

[0028] The complete action that ultimately affects the environment It is composed of the individual actions of all intelligent agents.

[0029] Node time The amount of reactive power injected is denoted as The reactive power adjustment required in the next time step in response to renewable energy fluctuations and load changes can be expressed by the following formula: in, This represents the inverter adjustment value, i.e., the action that the agent needs to perform.

[0030] Furthermore, the goal of multi-agent reinforcement learning algorithms is to derive a cooperative strategy that minimizes operating costs and network losses while maintaining voltage stability. Total reward value The negative sum, consisting of three independent penalty terms, is expressed as: in, This represents the voltage over-limit penalty item. This indicates a network loss penalty. This is the operating cost penalty item, with a weighting coefficient. , and These are key hyperparameters used to adjust the scale of these penalty terms.

[0031] Specifically, the voltage over-limit penalty, as a core penalty item, is designed to strictly prohibit voltage deviations from the safe operating range. The penalty value is calculated as follows: in, Indicates at time step Located at node phase The actual voltage measurement value. Represents a node The upper limit of voltage safety, Represents a node The lower limit of voltage safety.

[0032] Specifically, the network loss penalty term aims to guide the agent to seek solutions that can both stabilize voltage and improve the overall operating efficiency of the power grid. Total active power loss. This value, at the physical level, represents the sum of all resistive losses, primarily including the total resistance of all conductors. Losses and transformer losses. This penalty is the loss value itself.

[0033] Specifically, the operating cost penalty aims to avoid excessive or unnecessary use of reactive power resources, thereby promoting a more efficient and stable control strategy. It is calculated by summing the absolute values ​​of the normalized actions taken by all agents: in, The range is indicated in The normalization action within.

[0034] In this embodiment of the invention, step S200 optimizes the operation state of the distribution network within the voltage safety region by integrating a multi-objective reward mechanism that incorporates voltage safety constraints. This includes: constructing a fully cooperative multi-agent reinforcement learning system, wherein each agent generates reactive power adjustment actions based on local observations and shares a unified reward function. Before executing an action, the execution network performs a voltage safety check on the current system state. If the voltage of any node exceeds the voltage safety threshold, the current training or control process is immediately terminated. Based on the degree of deviation of the voltage of each node from the voltage safety threshold, the operating state is divided into three regions. When the voltage is not greater than the first voltage threshold and not less than the second voltage threshold, the current operating range is determined to be the first safe zone. When the voltage is less than the second voltage threshold and not less than the third voltage threshold, or the voltage is not greater than the fourth voltage threshold and greater than the first voltage threshold, the current operating range is determined to be the second safe zone. When the voltage is less than the third voltage threshold or greater than the fourth voltage threshold, the current operating range is determined to be a danger zone.

[0035] Furthermore, the policy generated by the execution network must satisfy... Constraints. Before executing an action, the execution network needs to determine its current state value. To ensure operational safety, the hard constraint function is designed as follows: in, Indicates the state of the system. Indicates the state Next, node The actual voltage measurement value, Indicates the reference voltage.

[0036] This constraint function aims to ensure that the strategy optimizes towards voltage stability. Its penalty function is designed as follows: Specifically, when the voltage In safe zone 1 At that time, the main goal of the intelligent agent is to minimize operating costs; when the voltage... In safe zone 2 At this time, the agent's primary objective is to minimize voltage deviation. It is particularly important to note that when the voltage... If the voltage safety is violated beyond the two feasible regions mentioned above, the training process of the agent will be forcibly terminated.

[0037] Furthermore, the state value function Defined in strategy The expected return is calculated using cumulative instant rewards: in, Mathematical expectation operator, subscript The expectation is that the agent follows the policy. It was calculated under the following circumstances. This represents the total time step of a training cycle. At time step Discount factor, Represents intelligent agents At time step The instant reward received.

[0038] Therefore, minimizing voltage violations and control costs will result in higher rewards and a better value assessment. Based on this, an expected cumulative reward function with hard constraints is designed.

[0039] In this embodiment of the invention, step S200 further includes: When the system is in the first safe zone, the reward function primarily minimizes the operating cost; when the system is in the second safe zone, the reward function primarily minimizes the voltage deviation. The expected cumulative reward function under hard constraints is defined as follows: in, For strategy The expected cumulative reward is as follows: Let the state value loss function be... For the KL divergence term, Let be the joint action consisting of the actions of all agents at time t.

[0040] It should be noted that, through inequalities This enforces the condition that the agent must operate within a preset safe voltage boundary. KL divergence term. As a constraint on the trust zone, it restricts the new policy from the current behavior. The degree of deviation in the distribution ensures the stability and conservatism of policy updates. The objective function aims to maximize rewards within the safe region while minimizing security risks within the feasible region.

[0041] For example, in safety region 1, the voltage is within an acceptable range. The control objective here is to minimize control costs while maintaining stability: For example, in safety region 2, the voltage has exceeded the preferred range. The primary objective here is to restore the voltage to the safe range, even at the cost of increased control complexity, leading to a shift in optimization focus: In this embodiment of the invention, step S200 further includes: Each optimal joint strategy satisfy: Furthermore, the value function can be derived from the state-action value function. It can be represented in the following form: In this embodiment of the invention, step S300, under the centralized training and distributed execution framework, includes introducing a cooperative advantage function, which includes: The cooperative advantage function is expressed as follows: in, To remove intelligent agents The actions of all other intelligent agents, For intelligent agents The local state value function.

[0042] It should be noted that, in order to guide the agent to learn the optimal policy within the safe voltage region, a cooperative advantage function is introduced, which reflects the difference between the joint Q value and the local state value. This function achieves cooperative behavior under a fully distributed control scheme by measuring the relative contribution of a single agent to the overall system performance.

[0043] In an optional embodiment, each agent is assumed to be... Define the collaborative advantage function If each agent makes choices in each local decision step that make the agent... Maximize the action By updating its strategy, the joint strategy of the multi-agent system will gradually converge to the global optimum as the local strategy is iteratively improved.

[0044] The optimal strategy for safe zone 1 is revised as follows: The optimal strategy in safe zone 2 is revised as follows: Policy updates based on cooperative advantage functions can ensure that the updated policy... Compared to the current strategy This achieves performance improvement. When all agents select actions according to this update rule, their joint action value function satisfies the following condition: in, This represents the value function of the joint action of a multi-agent system. Representing intelligent agents Updated strategy This represents the original strategy before the update.

[0045] This inequality shows that each local policy update increases the individual contribution of the agents and may improve the joint Q-value. Therefore, when all agents update their actions according to the cooperative advantage rule, the joint Q-value function will exhibit a monotonically non-decreasing trend, which can be expressed as: It should be noted that, in order to quantify the degree of cooperation between nodes, this invention introduces a joint metric as a cooperation measurement benchmark. Q Value function. This collaborative metric is denoted as It is used to assess the level of cooperation among agents and guide policy optimization.

[0046] Specifically, it is assumed that each agent follows a deterministic policy. Their joint action vector can be expressed as .

[0047] In this embodiment of the invention, step S300, which quantifies the contribution of each intelligent agent and its controlled communication switch and segment switch actions to the global performance to form a cooperative control strategy, includes: Define a cooperation metric for agents to evaluate their contribution to the overall system performance. The cooperation metric is expressed as: in, Indicates the presence of intelligent agents Total reward during the action, Indicates the exclusion of intelligent agents Total reward during the action, The larger the value, the stronger the agent. The higher the contribution to the overall system performance.

[0048] Furthermore, The larger the value, the stronger the agent. A higher contribution to the overall system performance reflects a stronger level of collaboration. in, The action value function represents the action of each agent.

[0049] It should be noted that the partial derivative relationship between the joint action value function and the individual action value function can effectively guide the parameter updates of the commentator hybrid network, thereby ensuring the balanced coordination of the contributions of all nodes.

[0050] In this embodiment of the invention, step S300 associates the actions of the physical devices controlled by the agent with their strategy output. The physical devices include feeder interconnection switches, transformer interconnection switches, substation interconnection switches, and sectionalizing switches. By analyzing the changes in system operating indicators before and after the switch action, the contribution value of the switch to photovoltaic absorption capacity and load power supply capacity in a specific scenario is quantified, and the contribution value is mapped to the corresponding agent's cooperation metric. Construct an objective function that integrates collaborative metrics to guide agent policy updates: in, This is an adjustment coefficient used to control the weight of the collaboration metric in the objective function; During the intensive training phase, each agent can acquire global state information and use the objective function as the policy gradient objective to update its agent network parameters. When the cooperation metric meets a preset threshold, the policy tends to select the cooperative action that is most beneficial to the system as a whole. The reward function is modified to depend not only on the individual Q-values ​​of each agent but also on the degree of cooperation between agents. This collaboration metric can effectively guide agents to choose more collaborative actions, thereby improving the overall benefits of the system.

[0051] It should be noted that this optimization objective not only pursues maximizing global rewards but also encourages agents to adopt more collaborative behavioral strategies during the learning process. Under the centralized training and distributed execution framework, each agent can acquire global information during the training phase and incorporates collaboration metrics into the policy update process. Policy updates are primarily guided by both the joint Q-value and the collaboration metric. This is achieved by setting a collaboration threshold. When the collaboration metric satisfies In this case, the generated strategy will tend to select the cooperative action that is most beneficial to the overall system, thereby achieving a dual optimization of system performance and cooperative efficiency.

[0052] Furthermore, intelligent agents The optimal strategy can be expressed as: The Bellman optimal equation under this cooperation metric is defined as: The optimal value function satisfies: Furthermore, by iteratively applying policy evaluation and improvement steps guided by collaborative metrics, the optimal policy can be obtained. Each agent's agent network updates its policy parameters using the modified reward function as the policy gradient objective to maximize the expected reward. The critic network updates its parameters by minimizing the Q-value prediction error, and its joint loss function is: in, b This indicates the batch size, representing the sampling of multiple trajectories from the experience replay buffer for synchronous training; The formula for calculating the Q-value target is: in, This indicates the revised global reward. and These are the parameters for the critic network and the critic hybrid network, respectively. and These correspond to the parameters of the target critic network and the target critic hybrid network.

[0053] Iteratively evaluate and improve the execution strategy until convergence is achieved to obtain the optimal collaborative control strategy.

[0054] It should be noted that this invention proposes a feeder group architecture concept, which significantly reduces the dimensionality of solving complex network reconfiguration problems by decoupling large-scale distribution networks into several feeder groups. Based on a hierarchical decision-making mechanism of first intra-group autonomy and then inter-group collaboration, an efficient reconfiguration control system is constructed. This can effectively support key application scenarios such as rapid load restoration in non-faulty sections under fault conditions and intelligent load transfer during peak summer / winter periods, ultimately achieving the operational optimization goal of rapid power restoration during faults and fewer power outages during peak hours.

[0055] Furthermore, by adjusting the combined states of the feeder interconnection switch, transformer interconnection switch, substation interconnection switch, and sectionalizing switch, precise power flow control at three levels—feeder-transformer-substation—can be achieved. The specific topology is as follows: Figure 2 As shown, between two feeders within the same transformer power supply range, across different transformer power supply ranges, and across different substation power supply ranges, flexible transfer of net load is achieved, forming a hierarchical and zoned power flow optimization capability.

[0056] In this embodiment of the invention, step S400, which dynamically adjusts the states of the multi-agent interconnection switch and the segmented switch based on the cooperative control strategy to realize multi-level load transfer, includes: establishing a multi-period reconfiguration model, wherein the multi-period reconfiguration model includes power flow constraints, security constraints, radial network topology constraints, photovoltaic power generation constraints, and load shedding constraints. By utilizing a multi-agent system to coordinate the interactions between different levels, when an overload or voltage limit is exceeded in any area, a corresponding switching action is triggered. Lower-level resources are prioritized for local adjustments. If the problem cannot be solved, higher-level resources are activated to participate in regulation. Based on the quantitative analysis results of the switch contribution, a long-term optimization strategy is formulated, and the action rules of each agent and its control device are regularly evaluated and adjusted.

[0057] Specifically, the power flow constraints are expressed as follows: in, Indicates time Inject node Total active power, Indicates time Inject node Total reactive power, Indicates time ,node voltage amplitude, Represents the set of all branches in the network. Indicates time Adjacent nodes voltage amplitude, Indicates a branch electrical conductivity, Indicates a branch susceptivity, Indicates time ,node and nodes The voltage phase angle difference between them.

[0058] Safety constraints are expressed as follows: in, Indicates time Flowing through the side road The magnitude of the current, and Representing branches The lower and upper limits of the permissible current. This represents the set of non-switch branches. Indicates the connection node and nodes The side road, For a binary decision variable, when This indicates that the switch is in the closed state. This indicates that the switch is in the off state. This represents the set of all operable switch branches. Indicates time ,node voltage amplitude, and Represents a node Permissible lower and upper voltage limits.

[0059] The topological constraints of a radial network are represented as follows: in, This represents the total number of nodes in the distribution network. This represents the number of substations. This represents the number of branches in the network that are always in a closed state. Represented as a branch road Virtual active power on For a very small virtual power injection value, Represented as nodes The set of child nodes of the parent node. Represented as nodes The set of parent nodes, Represented as a branch road The maximum virtual power allowed to flow through.

[0060] It should be noted that the distribution network maintains a radial structure, ensuring that the number of open and closed switches meets the distribution network topology constraints. This ensures the overall connectivity of the distribution network, guaranteeing that all loads are powered by virtual power sources, effectively preventing isolated nodes or operational islands within the distribution network.

[0061] Photovoltaic power generation constraints are expressed as follows: in, Indicates time Located at node The actual active power value that photovoltaic power generation units are allowed to inject into the distribution network. In time ,node The lower limit of the allowable output of the photovoltaic power generation unit. In time ,node The maximum active power that a photovoltaic power generation unit can produce.

[0062] The load shedding constraint is expressed as: in, In time Located at node The original active power demand of the load, Indicates time The actual supply nodes of the system The active power value of the load, node The load reduction ratio coefficient.

[0063] In an optional embodiment, the quantitative analysis of switch contributions requires a large sample size. A Monte Carlo method is used to generate... D This study focuses on typical scenarios, particularly considering photovoltaic power generation reduction and load shedding. Corresponding samples are obtained by optimizing network reconfiguration schemes for each scenario. Traditional network reconfiguration optimization methods are slow to solve, making large-scale sample generation extremely time-consuming.

[0064] Controllable switches are divided into three reconfiguration levels in ascending order: feeder level, transformer level, and substation level. Feeder level reconfiguration only involves the operation of controllable switches between feeders, and the same applies to the transformer and substation levels. During optimization, lower-level reconfigurations are prioritized, decomposing the global reconfiguration problem into several local reconfiguration sub-regions, thereby reducing the number of controllable switches involved in a single optimization. This reduces the dimensionality of binary variables in the reconfiguration problem, significantly improving the solution speed while avoiding wasted computational resources compared to global reconfiguration optimization, thus making it suitable for efficient sample collection.

[0065] The contribution of each switch is quantitatively evaluated. (For the sample) M The specific quantitative methods for photovoltaic absorption capacity and power supply capacity indicators are as follows: in, In time A certain substation The proportion of curtailed solar power in the region to the total daily curtailment in the region. In time A certain substation The proportion of load reduction in the area under its jurisdiction to the total daily load reduction in the area. It belongs to a substation The set of all photovoltaic nodes, It belongs to a substation The set of all load nodes, In time ,node The amount of solar power curtailment, In time ,node The amount of load reduction.

[0066] In obtaining and Then, the 24-hour power supply capacity can be quantified using the following formula: in, In a specific sample, the switch In time The contribution of a single action to photovoltaic power consumption. In a specific sample, the switch In time The contribution of a single action to the power supply to the load.

[0067] In obtaining the switch ij exist D In the samples and Then, the photovoltaic absorption capacity and power supply capacity of the switch can be obtained by the following formula: in, switch In all The average contribution to photovoltaic power consumption in each sample. switch In all The average contribution to load power supply in each sample. Total number of samples generated by methods such as Monte Carlo. In the sample In the middle, switch In time Contribution to photovoltaic power consumption In the sample In the middle, switch In time The contribution value to the power supply to the load. switch The final overall contribution Weighting coefficients.

[0068] Specific results are as follows Figure 3As shown in the results, by filtering the switching contributions, the action space reduces decision complexity, avoids overload or voltage exceedance risks caused by invalid switching operations, and improves proactive risk prevention capabilities. The training curves show that this method converges after approximately 1000 rounds, with a stable reward value of -180, while the SDPPG method exhibits large fluctuations in reward value, reaching only -250, indicating enhanced risk prediction capabilities. Compared to the SDPPG method, switching operation costs are reduced, resulting in lower overall operating costs. In fault scenarios, through hierarchical reconfiguration, load recovery time in non-faulty sections is shortened, the number of power outages during peak hours is reduced, and power supply reliability is improved.

[0069] Example 2: The above example is an illustrative scheme of a main grid load transfer method based on multi-agent cooperative optimization. It should be noted that the technical solution of this main grid load transfer system based on multi-agent cooperative optimization belongs to the same concept as the above-described main grid load transfer method based on multi-agent cooperative optimization. Details not described in detail in this example can be found in the description of the above-described main grid load transfer method based on multi-agent cooperative optimization.

[0070] This embodiment presents a main network load transfer system based on multi-agent cooperative optimization, comprising: The agent deployment module is used to divide the power distribution network into several feeder groups and deploy agents in each feeder group to build a multi-agent collaborative architecture. The multi-objective reward modeling module is used to model the multi-agent voltage stability problem as a distributed partially observable Markov decision process. By integrating the multi-objective reward mechanism with voltage safety constraints, the operation state of the distribution network is optimized within the voltage safety region. The strategy generation module is used to introduce a cooperative advantage function under the centralized training and distributed execution framework, quantify the contribution of each agent and its controlled contact switches and segmented switches to the global performance, and form a cooperative control strategy. The multi-level load transfer execution module is used to dynamically adjust the states of the multi-agent interconnection switch and segment switch based on the aforementioned collaborative control strategy, thereby realizing multi-level load transfer.

[0071] This embodiment also provides an electronic device applicable to a main network load transfer method based on multi-agent cooperative optimization, including: The system includes a memory and a processor. The memory stores computer-executable instructions, and the processor executes these instructions to implement the main network load transfer method based on multi-agent cooperative optimization proposed in the above embodiments.

[0072] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the main network load transfer method based on multi-agent cooperative optimization as proposed in the above embodiments.

[0073] The storage medium proposed in this embodiment and the main network load transfer method based on multi-agent cooperative optimization proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0074] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0075] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A main grid load transfer method based on multi-agent cooperative optimization, characterized in that, include: The power distribution network is divided into several feeder groups, and intelligent agents are deployed in each feeder group to build a multi-agent collaborative architecture. The multi-agent voltage stability problem is modeled as a distributed partially observable Markov decision process. By integrating a multi-objective reward mechanism with voltage safety constraints, the operation state of the distribution network is optimized within the voltage safety region. Under the framework of centralized training and distributed execution, a cooperative advantage function is introduced to quantify the contribution of each agent and its controlled contact switches and segmented switches to the global performance, thereby forming a cooperative control strategy. Based on the aforementioned collaborative control strategy, the states of multi-agent interconnection switches and segmented switches are dynamically adjusted to achieve multi-level load transfer.

2. The main network load transfer method based on multi-agent cooperative optimization as described in claim 1, characterized in that, The method of modeling the multi-agent voltage stability problem as a distributed partially observable Markov decision process includes: defining the state space, observation space, joint action space, state transition function, shared reward function, and discount factor; The state space is composed of environmental state and action-related state. The environmental state includes the active power injection and reactive power injection of each renewable energy node at the current time step, as well as the active power demand and reactive power demand of each load node. The action-related state includes the active power, reactive power, reactive power compensation value, transformer tap position, and node voltage amplitude flowing into each node. Each agent can only observe local state information related to its electrical location, including the voltage amplitude, power injection, load demand, and renewable energy output of its node. The combined action space output is a normalized reactive power adjustment command, used to control the reactive power compensation device; the actions of all agents are spliced ​​together to form a combined action and act on the environment; The shared reward function is the negative sum of voltage over-limit penalty, network loss penalty, and operating cost penalty.

3. The main network load transfer method based on multi-agent cooperative optimization as described in claim 2, characterized in that, By integrating a multi-objective reward mechanism that incorporates voltage safety constraints, the operation of the distribution network within the voltage safety zone can be optimized by: constructing a fully cooperative multi-agent reinforcement learning system, in which each agent generates reactive power regulation actions based on local observations and shares a unified reward function; Before executing an action, the execution network performs a voltage safety check on the current system state. If the voltage of any node exceeds the voltage safety threshold, the current training or control process is immediately terminated. Based on the degree of deviation of the voltage of each node from the voltage safety threshold, the operating state is divided into three regions. When the voltage is not greater than the first voltage threshold and not less than the second voltage threshold, the current operating range is determined to be the first safe zone. When the voltage is less than the second voltage threshold and not less than the third voltage threshold, or the voltage is not greater than the fourth voltage threshold and greater than the first voltage threshold, the current operating range is determined to be the second safe zone. When the voltage is less than the third voltage threshold or greater than the fourth voltage threshold, the current operating range is determined to be a danger zone.

4. The main network load transfer method based on multi-agent cooperative optimization as described in claim 3, characterized in that, Also includes: When the system is in the first safe zone, the reward function primarily aims to minimize the operating cost; When the system is in the second safe zone, the reward function primarily minimizes the voltage deviation; The expected cumulative reward function under hard constraints is defined as follows: in, For strategy The expected cumulative reward is as follows: Let the state value loss function be... For the KL divergence term, Let be the joint action consisting of the actions of all agents at time t. Each optimal joint strategy satisfy:

5. The main network load transfer method based on multi-agent cooperative optimization as described in claim 4, characterized in that, In the framework of centralized training and distributed execution, the following collaborative advantage functions are introduced: The cooperative advantage function is expressed as follows: in, To remove intelligent agents The actions of all other intelligent agents, For intelligent agents The local state value function.

6. The main network load transfer method based on multi-agent cooperative optimization as described in claim 5, characterized in that, The contribution of each intelligent agent and its controlled interconnecting and segmenting switches to global performance is quantified to form a cooperative control strategy, including: Define a cooperation metric for agents to evaluate their contribution to the overall system performance. The cooperation metric is expressed as: in, Indicates the presence of intelligent agents Total reward during the action, Indicates the exclusion of intelligent agents Total reward during the action, The larger the value, the stronger the agent. The higher the contribution to the overall system performance; The actions of the physical devices controlled by the intelligent agent are associated with its policy output. The physical devices include feeder tie switches, transformer tie switches, substation tie switches, and sectionalizing switches. By analyzing the changes in system operating indicators before and after the switch action, the contribution value of the switch to the photovoltaic absorption capacity and load power supply capacity in a specific scenario is quantified, and the contribution value is mapped to the corresponding intelligent agent's collaborative metric. Construct an objective function that integrates collaborative metrics to guide agent policy updates: in, This is an adjustment coefficient used to control the weight of the collaboration metric in the objective function; During the intensive training phase, each agent can acquire global state information and use the objective function as the policy gradient objective to update its agent network parameters. When the cooperation metric meets a preset threshold, the policy tends to select the cooperative action that is most beneficial to the system as a whole. The critic network updates its parameters by minimizing the Q-value prediction error, and its joint loss function is: in, b This indicates the batch size, representing the sampling of multiple trajectories from the experience replay buffer for synchronous training; Iteratively evaluate and improve the execution strategy until convergence is achieved to obtain the optimal collaborative control strategy.

7. The main network load transfer method based on multi-agent cooperative optimization as described in claim 6, characterized in that, Based on the aforementioned collaborative control strategy, the states of multi-agent interconnection switches and segmented switches are dynamically adjusted to achieve multi-level load transfer, including: establishing a multi-period reconfiguration model, which includes power flow constraints, security constraints, radial network topology constraints, photovoltaic power generation constraints, and load shedding constraints. By utilizing a multi-agent system to coordinate the interactions between different levels, when an overload or voltage limit is exceeded in any area, a corresponding switching action is triggered. Lower-level resources are prioritized for local adjustments. If the problem cannot be solved, higher-level resources are activated to participate in regulation. Based on the quantitative analysis results of the switch contribution, a long-term optimization strategy is formulated, and the action rules of each agent and its control device are regularly evaluated and adjusted.

8. A main grid load transfer system based on multi-agent cooperative optimization, applied to the method described in any one of claims 1-7, characterized in that, include: The agent deployment module is used to divide the power distribution network into several feeder groups and deploy agents in each feeder group to build a multi-agent collaborative architecture. The multi-objective reward modeling module is used to model the multi-agent voltage stability problem as a distributed partially observable Markov decision process. By integrating the multi-objective reward mechanism with voltage safety constraints, the operation state of the distribution network is optimized within the voltage safety region. The strategy generation module is used to introduce a cooperative advantage function under the centralized training and distributed execution framework, quantify the contribution of each agent and its controlled contact switches and segmented switches to the global performance, and form a cooperative control strategy. The multi-level load transfer execution module is used to dynamically adjust the states of the multi-agent interconnection switch and segment switch based on the aforementioned collaborative control strategy, thereby realizing multi-level load transfer.

9. An electronic device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the main network load transfer method based on multi-agent cooperative optimization as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the main network load transfer method based on multi-agent cooperative optimization as described in any one of claims 1 to 7.

Images