Multi-agent and energy storage collaborative optimization method and system for eliminating power grid section card hole

Abstract

A method and system for eliminating power grid section card slots through multi-agent and energy storage cooperation optimization, the method comprising: constructing a three-layer multi-agent reinforcement learning framework comprising a system coordination agent, a DC engineering agent and an energy storage agent; calculating a power flow sensitivity matrix of a target power grid section set on each node injection power, and screening the energy storage candidate nodes based on the power flow sensitivity matrix to obtain a candidate node set; using a proximal policy optimization algorithm to cooperatively optimize the energy storage configuration capacity of each candidate node in a new energy output uncertainty scenario set to obtain an energy storage configuration capacity result; training the three-layer multi-agent reinforcement learning framework to output original control actions, and mapping the original control actions to a feasible region that meets the constraint conditions through orthogonal projection to output feasible control actions. The present application has comprehensive superiority in terms of section card slot elimination effect, safety constraint guarantee, multi-DC engineering coordination control adaptive ability and energy storage resource utilization efficiency.

Description

Technical Field

[0001] This invention relates to the field of power system optimization and control technology, and in particular to a method and system for eliminating grid cross-section bottlenecks through multi-agent and energy storage collaborative optimization. Background Technology

[0002] The large-scale integration of new energy sources, represented by photovoltaics and wind power, has become a core characteristic of the construction of new power systems. With a high proportion of renewable energy connected to the grid, the penetration rate of power electronic equipment continues to increase. The strong randomness, anti-peak-shaving, and weak support of new energy output have led to a complex power flow distribution pattern in the grid, characterized by rapid fluctuations, large-scale transfers, and bidirectional flows—a phenomenon never seen in traditional AC systems. Simultaneously, the dense integration of multiple high-voltage direct current (HVDC) projects further exacerbates the difficulty of power flow regulation in regional power grids: the large-scale, deterministic transfer of DC power, while effectively supporting inter-regional energy transmission, also generates significant additional disturbances to the power flow distribution at AC sections. The combined effect of these two factors has led to a significant increase in the bottleneck risk at key transmission sections of the power grid. Actual measurement data shows that, under a high proportion of renewable energy scenarios, the power flow standard deviation of key interconnect lines during a typical operating day can expand from ±15% of the historical baseline to ±22%, an increase of 47%. The frequency of section overload events also shows a significant upward trend, seriously threatening the safe and stable operation of the power system.

[0003] Cross-sectional bottleneck elimination is a crucial means of ensuring the safe and stable operation of power grids with a high proportion of power electronics. Its core lies in controlling the real-time power flow at critical cross-sections within the thermal stability limit and maximizing the safety margin through the coordinated optimization of various controllable resources (including DC power modulation, energy storage charging and discharging, adjustable loads, and renewable energy output regulation). However, existing cross-sectional bottleneck elimination technologies have the following main shortcomings:

[0004] (1) The coordinated control strategy for multiple DC projects lacks adaptive capability. Existing coordinated control schemes for multiple DC projects generally adopt offline tuning rules based on operational experience, with fixed control parameters. They cannot adaptively adjust the power allocation scheme of each DC project online according to the real-time fluctuations in renewable energy output. In scenarios with a high proportion of renewable energy, due to the obvious "duck curve" characteristic of photovoltaic output, the amplitude and direction of power flow changes at different times of the day vary significantly. Fixed-rule control methods are difficult to effectively cover all operating conditions, resulting in unstable control effects and a high frequency of cross-section overruns.

[0005] (2) The configuration decisions and operation strategies of energy storage systems are disconnected, and the degree of synergistic optimization is insufficient. Existing energy storage optimization research usually treats site selection and capacity determination (planning layer) and charge and discharge control (operation layer) as relatively independent problems to be solved separately, and there is a lack of effective information feedback mechanism between the two layers. This fragmented optimization approach ignores the strong coupling relationship between energy storage deployment location, configuration capacity and real-time control effect, so that the energy storage configuration scheme determined at the planning layer cannot achieve the expected cross-sectional support effect at the operation layer, and the utilization rate of energy storage resources is generally low.

[0006] (3) There is a lack of a unified and collaborative optimization framework that takes into account coupling constraints for various types of regulation resources. Existing methods usually design photovoltaic power output regulation, DC power modulation, and energy storage charging and discharging as independent control links, and lack the ability to perform global collaborative optimization under a unified framework of power balance constraints, equipment safety constraints, and cross-sectional capacity constraints. There may be problems of mutual cancellation or improper superposition of regulation effects among various regulation methods, making it impossible to achieve Pareto optimal cross-sectional management at the whole grid level.

[0007] (4) Lack of a safety and feasibility guarantee mechanism for reinforcement learning methods. Reinforcement learning (RL) methods have significant advantages in dealing with high-dimensional, nonlinear, and uncertain decision-making problems, but their direct application to power system regulation faces key challenges: the regulation actions output by neural networks cannot inherently guarantee that power balance constraints and equipment safety constraints are met. Existing research usually uses penalty terms to softly constrain feasibility, but this cannot fundamentally eliminate the possibility of constraint violations, which severely limits the engineering application of reinforcement learning in practical power dispatching systems.

[0008] (5) The computational scalability of energy storage capacity optimization under large-scale uncertainty scenarios is insufficient. When solving the large-scale energy storage location and capacity determination problem, the computational complexity of the classic mixed integer programming (MIP) method increases exponentially with the increase of the grid node size and the number of uncertainty scenarios. It is difficult to complete the solution in an acceptable time under large-scale grid engineering scenarios, which restricts its engineering practicality.

[0009] Therefore, there is an urgent need for an efficient method to eliminate cross-sectional bottlenecks that can achieve collaborative optimization of multiple DC projects and energy storage systems, have a strict safety constraint guarantee mechanism, and can adapt to the rapid fluctuation characteristics of power flow in high-proportion new energy scenarios. Summary of the Invention

[0010] The purpose of this invention is to overcome the above-mentioned defects and problems in the prior art and provide a method and system for eliminating grid section bottlenecks through multi-agent and energy storage collaborative optimization. This addresses issues such as the lack of adaptive capability in the coordinated control of multiple DC projects, the fragmentation of energy storage configuration and operation strategies, the lack of a unified collaborative framework for multiple types of regulation resources, the difficulty in ensuring the safety and feasibility of reinforcement learning actions, and insufficient computational efficiency in large-scale uncertainty scenarios.

[0011] To achieve the above objectives, the technical solution of the present invention is:

[0012] In a first aspect, the present invention provides a method for eliminating grid cross-section bottlenecks through multi-agent and energy storage collaborative optimization, comprising:

[0013] Real-time operation data of the target power grid is collected, and a three-layer multi-agent reinforcement learning framework is constructed, which includes a system coordination agent, a DC engineering agent, and an energy storage agent. The system coordination agent is used to characterize global operation information, the DC engineering agent is used to characterize the local adjustability and operation constraints of the DC engineering, and the energy storage agent is used to characterize the local adjustability and operation constraints of the energy storage unit.

[0014] Calculate the power flow sensitivity matrix of the target power grid section set to each node for injected power, and filter the energy storage candidate nodes based on the power flow sensitivity matrix to obtain the candidate node set;

[0015] A near-end strategy optimization algorithm is used to collaboratively optimize the energy storage configuration capacity of each candidate node in a set of uncertain new energy output scenarios to obtain the energy storage configuration capacity result;

[0016] The three-layer multi-agent reinforcement learning framework is trained to output the original control actions, and the original control actions are mapped to the feasible domain that satisfies the constraints through orthogonal projection, and the feasible control actions are output.

[0017] Preferably, the current flow sensitivity is:

[0018] ;

[0019] In the formula, Represents a node The injected active power on the cross section The sensitivity of the active current flow; Indicates cross-section The corresponding branch aggregation coefficient vector; This represents the diagonal matrix formed by the branch reactances; Represents the branch-node correlation matrix; This represents the nodal susceptance matrix after deducting the equilibrium point; Represents a node The unit injection vector; This represents the reference vector corresponding to the balancing node.

[0020] Preferably, the step of filtering candidate energy storage nodes based on the power flow sensitivity matrix to obtain a candidate node set includes:

[0021] For each cross section, the absolute value of the tidal current sensitivity is used. Sort the nodes from largest to smallest, and use the cumulative coverage criterion to determine the number of candidate nodes for each section. Then, take the union of the candidate nodes for each section to form the final candidate node set. ;

[0022] The number of candidate nodes for each cross section is:

[0023] ;

[0024] In the formula, Indicates cross-section The number of candidate nodes selected; Indicates by The th in descending order Index of each node; This indicates the preset cumulative sensitivity coverage threshold. .

[0025] Preferably, the reward function of the near-end policy optimization algorithm is:

[0026] ;

[0027] ;

[0028] In the formula, express The reward value at any given moment; Indicates cross-section The severity index of the checkpoint; Indicates the first Energy storage configuration capacity corresponding to each candidate node; This represents the equivalent investment cost per unit capacity of energy storage. This indicates that the safety constraints satisfy the indicated quantity; , , These are the weighting factors, used to balance the safety of the cross section, investment costs, and the degree to which feasibility constraints are met. Indicates cross-section Positive load rate, , Indicates cross-section Real-time active power flow, Indicates cross-section The permissible limits under positive current; Indicates cross-section Reverse load rate, , Indicates cross-section Permissible limits under reverse current.

[0029] Preferably, the training three-layer multi-agent reinforcement learning framework outputs the original control actions, including:

[0030] During the intensive training phase, all agents share the global value evaluator, and the global value function is denoted as... ,in, Represents the global state; This represents a joint action consisting of the actions of all agents, expressed through the global state. With joint actions A unified assessment is conducted to identify the comprehensive impact of each local action on the overall cross-sectional safety target;

[0031] During the distributed execution phase, the system coordination agent generates a global coordination boundary based on the target grid section risk level, the ranking results of the absolute value of power flow sensitivity, and the energy storage configuration capacity results. Under the constraints of the global coordination boundary, the DC engineering agent learns the coordination modulation strategy between multiple DC circuits, and the energy storage agent learns the local fast charging and discharging strategy under the constraints of the global coordination boundary.

[0032] Preferably, during the training process of the three-layer multi-agent reinforcement learning framework, the first... The advantage function of each agent is:

[0033] ;

[0034] In the formula, Indicates the first Each agent in the global state Joint actions The dominant function under these conditions; Indicates except the first The set of actions of the other intelligent agents outside the intelligent body; Indicates the first The baseline action or reference action of an agent.

[0035] Preferably, the step of mapping the original control action to a feasible domain that satisfies the constraints through orthogonal projection and outputting a feasible control action includes:

[0036] ;

[0037] ;

[0038] In the formula, Indicates feasible regulatory actions; This represents the feasible region that satisfies power balance constraints, equipment output boundary constraints, equipment ramp-up constraints, and energy storage operation constraints. This represents a combined action consisting of the actions of all agents. Indicates the original regulatory action; This represents the power vector of each DC project regulation; Represents the new energy regulation power vector; This represents the energy storage charging and discharging power vector.

[0039] Preferably, when the update trigger condition is met, the advantage function and policy pruning objective are calculated, and the parameters of the three-layer multi-agent reinforcement learning framework are incrementally updated. The policy pruning objective is expressed as:

[0040] ;

[0041] In the formula, This indicates that the current policy and the sampling policy are in... The probability ratio at time t; Indicates the current policy network parameters; This represents the advantage function estimated from trajectory samples, which consist of the state, action, reward, and the projection correction of the original control action within the scheduling period. This indicates the pruning threshold for the strategy.

[0042] Secondly, the present invention provides a grid cross-section bottleneck elimination system for multi-agent and energy storage collaborative optimization, the system being used to implement the aforementioned grid cross-section bottleneck elimination method for multi-agent and energy storage collaborative optimization, the system comprising:

[0043] The three-layer MARL framework construction module is used to collect real-time operation data of the target power grid and construct a three-layer multi-agent reinforcement learning framework including a system coordination agent, a DC engineering agent, and an energy storage agent. The system coordination agent is used to characterize global operation information, the DC engineering agent is used to characterize the local adjustability and operation constraints of the DC engineering, and the energy storage agent is used to characterize the local adjustability and operation constraints of the energy storage unit.

[0044] The candidate node set acquisition module is used to calculate the power flow sensitivity matrix of the target power grid section set to each node, and to filter the energy storage candidate nodes based on the power flow sensitivity matrix to obtain the candidate node set.

[0045] The energy storage configuration capacity result acquisition module is used to perform collaborative optimization of the energy storage configuration capacity of each candidate node in a set of uncertain new energy output scenarios using a near-end strategy optimization algorithm to obtain the energy storage configuration capacity result.

[0046] The feasible control action acquisition module is used to train the three-layer multi-agent reinforcement learning framework to output the original control actions, and to map the original control actions to the feasible domain that meets the constraints through orthogonal projection, and output the feasible control actions.

[0047] Thirdly, the present invention provides a grid cross-section bottleneck elimination device for multi-agent and energy storage collaborative optimization, including a memory and a processor;

[0048] The memory is used to store computer program code and transmit the computer program code to the processor;

[0049] The processor is configured to execute, according to instructions in the computer program code, the grid cross-section bottleneck elimination method described above for multi-agent and energy storage collaborative optimization.

[0050] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0051] This invention presents a method and system for eliminating grid cross-section bottlenecks through multi-agent and energy storage collaborative optimization. The proposed three-layer multi-agent reinforcement learning framework coordinates the overall cross-section bottleneck elimination objective and constrains the action range of lower-layer agents. The DC engineering agent implements adaptive coordinated modulation of multiple DC power circuits, and the energy storage agent executes rapid charge and discharge commands to smooth residual fluctuations. The synergistic effect of the three-layer agents significantly outperforms single control methods. Furthermore, the power flow sensitivity screening results are not merely used to independently narrow the search space, but are further used as prior inputs for capacity optimization, node support features for the energy storage agent, and the basis for the system coordination agent to generate resource boundaries, thus forming a forward-coupled link of "sensitivity screening - capacity configuration - hierarchical control." Simultaneously, the innovatively designed constraint satisfaction layer rigorously projects the original control actions to the feasible region by solving a quadratic programming problem online, and feeds back the projection correction and constraint activity information to the online update process, forming a feedback loop of "strategy generation - security correction - parameter correction," solving the core challenge of security and feasibility in the deployment of reinforcement learning in actual power systems. In addition, the integrated energy storage site selection, capacity determination and collaborative control method proposed in this invention, based on cross-sectional aggregation sensitivity and PPO updated in the same strategy, compresses the optimization decision space to less than 30% of the total network node scale while effectively retaining high-value candidate locations, and improves the energy storage resource utilization efficiency through the coupling of the planning layer and the operation layer. Attached Figure Description

[0052] Figure 1 This is a flowchart of the grid section bottleneck elimination method proposed in this invention, which involves multi-agent and energy storage collaborative optimization.

[0053] Figure 2 This is a schematic diagram of the structure of the three-layer multi-agent reinforcement learning framework proposed in this invention.

[0054] Figure 3 This is a comparison chart of the standard deviation of cross-sectional power flow between the method proposed in this invention and the baseline method.

[0055] Figure 4 This is a comparison chart of the cross-sectional over-limit frequency between the method proposed in this invention and the baseline method.

[0056] Figure 5 This is a comparison chart of the training convergence curves of the method proposed in this invention and the baseline method.

[0057] Figure 6 This is the heat map of power flow sensitivity distribution proposed in this invention.

[0058] Figure 7 This is a schematic diagram showing the comparison results of energy storage optimization configurations under different schemes proposed in this invention.

[0059] Figure 8 This is a structural block diagram of the grid cross-section bottleneck elimination system proposed in this invention, which involves multi-agent and energy storage collaborative optimization.

[0060] Figure 9 This is a structural block diagram of the grid cross-section bottleneck elimination device proposed in this invention, which involves multi-agent and energy storage collaborative optimization. Detailed Implementation

[0061] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0062] See Figure 1 This invention provides a method for eliminating grid cross-section bottlenecks through multi-agent and energy storage collaborative optimization, comprising:

[0063] S1. Collect real-time operation data of the target power grid and construct a three-layer multi-agent reinforcement learning (MARL) framework that includes a system coordination agent, a DC engineering agent, and an energy storage agent. The system coordination agent is used to characterize global operation information, the DC engineering agent is used to characterize the local adjustability and operation constraints of the DC engineering, and the energy storage agent is used to characterize the local adjustability and operation constraints of the energy storage unit.

[0064] S2. Calculate the power flow sensitivity matrix of the target power grid section set to each node, and filter the energy storage candidate nodes based on the power flow sensitivity matrix to obtain the candidate node set.

[0065] S3. The Proximal Policy Optimization (PPO) algorithm is used to collaboratively optimize the energy storage configuration capacity of each candidate node in the set of uncertain new energy output scenarios to obtain the energy storage configuration capacity result.

[0066] S4. Train the three-layer multi-agent reinforcement learning framework to output the original control actions, and map the original control actions to the feasible domain that satisfies the constraints through orthogonal projection, and output the feasible control actions.

[0067] The proposed grid section bottleneck elimination method based on multi-agent and energy storage collaborative optimization is particularly suitable for the safety margin management and bottleneck elimination problems of key transmission sections in scenarios with high proportion of new energy access and dense feed-in of multiple DC projects. It addresses issues such as the lack of adaptive capability in the coordinated control of multiple DC projects, the disconnect between energy storage configuration and operation strategies, the lack of a unified collaborative framework for multiple types of regulation resources, the difficulty in ensuring the safety and feasibility of reinforcement learning actions, and insufficient computational efficiency in large-scale uncertain scenarios. The proposed three-layer multi-agent reinforcement learning framework uses a system coordination agent to coordinate the global bottleneck elimination target and constrain the action range of lower-layer agents. The DC project agent implements adaptive coordinated modulation of multiple DC power lines, and the energy storage agent executes rapid charge and discharge commands to smooth residual fluctuations. The collaborative effect of the three-layer agents is significantly better than a single regulation method. Furthermore, the power flow sensitivity screening results are not only used to independently narrow the search space but also serve as prior inputs for capacity optimization, node support features for energy storage agents, and the basis for the system coordination agent to generate resource boundaries, thus forming a forward-coupled link of "sensitivity screening - capacity configuration - hierarchical control". Meanwhile, the constraint satisfaction layer of this invention innovatively projects MARL control actions strictly onto the feasible region by solving the quadratic programming problem online, and feeds back the projection correction amount and constraint activity information to the online update process, forming a feedback loop of "policy generation-security correction-parameter correction", which solves the core problem of security feasibility in the deployment of reinforcement learning in actual power systems.

[0068] Further, see Figure 2 , Figure 2 This framework illustrates the hierarchical relationships and information interaction paths among the system coordination agent, DC engineering agent, and energy storage agent in a three-layer multi-agent reinforcement learning framework (three-layer MARL framework). The state space construction of the three-layer multi-agent reinforcement learning framework follows the modeling principle of "combining global coordination state and local execution state." Specifically, the system coordination layer characterizes global operational information such as cross-sectional safety, resource boundaries, and new energy fluctuations, while the local execution layers characterize the local adjustability and operational constraints of the DC engineering and energy storage units, respectively. This provides a unified, complete, and hierarchically callable state representation for subsequent collaborative optimization.

[0069] Specifically, real-time operational data is obtained from the energy management system, dispatch automation system, and synchronous phasor measurement device of the target power grid, and time alignment, anomaly removal, state fusion, and quality verification are performed on the multi-source data to form a unified data foundation that can be used for the construction of multi-agent states.

[0070] The collected data may include bus voltage, line power flow, DC project operating status, renewable energy output, energy storage status of charge, and constraint information related to safety verification, etc., to support subsequent status construction, section risk assessment, energy storage configuration optimization, and collaborative control decisions. The above data includes both network-wide operational information and dynamic status information at the local equipment level.

[0071] Based on the above operational status information, state vectors for the system coordination agent, the DC engineering agent, and the energy storage agent are constructed according to a three-layer multi-agent reinforcement learning framework. The system coordination agent is responsible for representing the global cross-sectional safety state and resource coordination boundary; its state can be denoted as... The DC engineering agent is responsible for characterizing the local adjustability and safety and stability-related states of the DC channel, and its states can be denoted as follows: The energy storage agent is responsible for characterizing the charge level, power boundary, and local support potential of the energy storage device; its state can be denoted as... This hierarchical state structure allows for the effective connection between global objectives and local controls.

[0072] Specifically, the state vector of the system coordinating agent , used to describe The network-wide observable operational status at any given time; It should include at least the following information: power grid topology and operation mode code. It is used to characterize the current network connectivity, operation mode switching status, and key equipment commissioning / decommissioning status; the load rate vector of each monitoring section Used to characterize the safety margin of a cross-section; a set of operating states for each DC project. It is used to characterize the current power level, operating boundary, and stability-related state of multiple DC transmission lines; and to predict the output of new energy sources. and its prediction error statistics This is used to characterize random fluctuations. The load rate of the monitoring section is denoted as... Its definition , Indicates the first Real-time active power flow at each monitoring section This indicates the maximum allowable transmission limit for this cross section. This represents the total number of monitored sections. Through the above construction, the system coordinating agent can perceive the safety pressure of sections and the adjustability margin of multiple resources from a global perspective.

[0073] Specifically, the first The state vector of a DC engineering intelligent agent Used to depict the first The partial operating status of the DC transmission project. This may include quantities such as sending and receiving end voltages, DC current, DC power, power change rate, commutation failure risk indicators, and reactive power support margin, among which, The DC project is numbered. Through local state description, the DC project agent can identify the adjustable range, dynamic response capability, and key constraints that may affect safety and stability of the current DC circuit at the current control moment, thereby outputting local power modulation actions consistent with the system coordination layer boundary.

[0074] Specifically, the first State vector of an energy storage agent Used to depict the first The local adjustability of each energy storage unit. At least the state of charge Maximum available charging power Upper limit of discharge power The risk characterization quantity of the corresponding cross-section and the local operational constraint information of the node, among which, The energy storage units are numbered. Through state vectors, the energy storage agent can sense both the local energy reserve and the charge / discharge boundary, as well as the supporting effect of its node on the critical section. Thus, during local rapid control, it prioritizes responding to the location most effective in eliminating the section bottleneck.

[0075] Furthermore, the construction of the power flow sensitivity matrix and the screening of candidate nodes are used to establish a quantitative mapping relationship between "node injection changes" and "target section power flow changes," and on this basis, identify key energy storage candidate nodes that significantly contribute to the elimination of section bottlenecks. The sensitivity results are not only used to narrow the search space for energy storage site selection and capacity optimization, but also serve as shared prior information for subsequent capacity optimization, energy storage agent state construction, and reward weight allocation, thereby forming a data coupling link between "candidate node screening - capacity configuration - online control."

[0076] Based on the DC power flow approximation of the power flow equations, the influence coefficients of the injected power at each node on the power flow of the target section are calculated. For a target section composed of several monitoring branches... ,node The cross-sectional tidal current sensitivity coefficient is expressed in the form of branch tidal current distribution factor aggregation, and its expression is as follows:

[0077] ;

[0078] In the formula, Represents a node The injected active power on the cross section The sensitivity of the active power flow; Indicates cross-section The corresponding branch aggregation coefficient vector is used to characterize each monitored branch that constitutes the cross section and its direction. For branches that belong to the cross section and whose power flow direction is consistent with the cross section's specified direction, the corresponding element of the vector is +1. For branches that belong to the cross section and whose power flow direction is opposite to the cross section's specified direction, the corresponding element of the vector is -1. For branches that do not belong to the cross section, the corresponding element of the vector is 0. This represents a diagonal matrix composed of reactances of relevant branches. Relevant branches refer to the set of transmission branches used to establish the branch-node correlation matrix and reactance matrix under the current operating mode, including at least the monitoring branches that constitute the target section. Represents the branch-node correlation matrix; This represents the nodal susceptance matrix after deducting the equilibrium point; Represents a node The unit injection vector; This represents the reference vector corresponding to the balancing node.

[0079] The above expression takes into account the location of the reference node, the distribution of branch reactance, and the combination relationship of the branch in the cross section. It can characterize the linear approximate mapping between the node injection change and the power flow change in the target cross section. It is the basis for subsequent candidate node selection, capacity configuration optimization and online control coupling modeling.

[0080] To narrow the search space for energy storage site selection and capacity optimization, each cross-section is calculated based on the absolute value of power flow sensitivity. Sort the nodes from largest to smallest, and use the cumulative coverage criterion to determine the number of candidate nodes for each section. Then, take the union of the candidate nodes for each section to form the final candidate node set. ;

[0081] The number of candidate nodes for each cross section is:

[0082] ;

[0083] In the formula, Indicates cross-section The number of candidate nodes selected; Indicates by The th in descending order Index of each node; This indicates the preset cumulative sensitivity coverage threshold. .

[0084] The above design significantly reduces the decision space for subsequent capacity optimization while maintaining the coverage capability of key sensitive nodes, thus improving the scalability of the method in large-scale power grid scenarios. It avoids blind searching across the entire network and ensures that capacity optimization is primarily focused on locations that contribute significantly to eliminating cross-sectional bottlenecks.

[0085] Furthermore, the collaborative optimization of energy storage capacity takes the set of uncertain scenarios in renewable energy output as input and jointly determines the set of candidate nodes through strategy optimization methods. The configuration capacity of each energy storage unit within the system. The goal of this step is not simply to allocate capacity to each node, but to establish a synergistic trade-off between "cross-sectional support effect, investment cost, and safety feasibility," and to directly couple the capacity optimization results to the subsequent energy storage node deployment, local action boundaries, and global reward construction in the three-layer MARL framework.

[0086] First, an uncertainty scenario set is constructed based on historical renewable energy output data, enabling the generated scenarios to cover typical fluctuation patterns, prediction biases, and trends in cross-sectional safety margin changes; then, a candidate node set is used... Using node characteristics, cross-sectional sensitivity information, and system operation constraints as inputs, the energy storage configuration capacity vector of each candidate node is solved. ,in, Indicates candidate nodes The corresponding energy storage configuration capacity. The capacity optimization process evaluates the improvement in cross-sectional power flow suppression effect after configuring energy storage capacity at each candidate node under different renewable energy fluctuation scenarios, and incorporates this improvement, along with investment costs, into a unified objective function. After capacity optimization, the sensitivity ranking results are compared with the capacity vector. A three-layer multi-agent reinforcement learning framework is constructed: firstly, it serves as the node support features and adjustable power limit for energy storage agents; secondly, it serves as the basis for the system coordinating agent to construct cross-sectional risk weights and resource allocation boundaries. To ensure that the capacity configuration results are consistent with the cross-sectional bottleneck elimination target, the following reward function is defined:

[0087] ;

[0088] In the formula, express The reward value at any given moment; Indicates cross-section The severity index of the checkpoint; Indicates the first Energy storage configuration capacity corresponding to each candidate node; This represents the equivalent investment cost per unit capacity of energy storage. This indicates that the safety constraint is satisfied. The safety constraint satisfaction indicator is obtained by the constraint satisfaction layer or the safety verification module based on the verification results of power balance constraints, equipment output boundary constraints, equipment ramping constraints, energy storage state of charge constraints, and cross-sectional power flow limit constraints. It is 1 when all constraints are satisfied and 0 when any constraint is not satisfied, or it is the proportion of scenarios that satisfy the safety constraints in scenario optimization. , , These are trade-off coefficients used to balance the degree to which cross-section safety, investment cost, and feasibility constraints are met. After normalizing the cross-section safety target, investment cost target, and feasibility constraint target, these trade-off coefficients are tuned according to the cross-section safety priority, investment cost constraints, and the requirement for exceeding the limit penalty, or obtained through cross-validation / grid search optimization.

[0089] During capacity optimization, the control objective is to strike a balance between cross-section bottleneck risks, energy storage investment costs, and safety feasibility, ensuring that the optimization results both enhance the support capacity of critical cross-sections and avoid unnecessary redundant energy storage configurations. In other words, capacity configuration must not only be "effective" but also "economical" and "feasible." The aforementioned reward function unifies cross-section safety benefits, capacity configuration costs, and engineering feasibility within the same optimization framework.

[0090] Cross-sectional checkpoint severity index It can be further defined as:

[0091] ;

[0092] In the formula, Indicates cross-section Positive load rate, , Indicates cross-section Real-time active power flow, Indicates cross-section The permissible limits under positive current; Indicates cross-section Reverse load rate, , Indicates cross-section Permissible limits under reverse current.

[0093] Since the flow of electricity at the cross-section may reverse in scenarios with a high proportion of new energy, this indicator considers both positive and negative over-limit situations, thus enabling a unified and complete measurement of cross-section bottleneck risks.

[0094] Specifically, the set of uncertain new energy output scenarios is obtained through existing scenario generation methods, including: acquiring historical output data, short-term predicted output, and prediction error statistics of wind and solar power in the target power grid; clustering the historical output curves on typical days or typical time periods; and performing Monte Carlo perturbation sampling based on the prediction error statistics to generate initial new energy output scenarios; and performing probability weighting and scenario reduction on the initial new energy output scenarios to obtain a set of uncertain new energy output scenarios for energy storage configuration capacity collaborative optimization.

[0095] Furthermore, the three-layer multi-agent reinforcement learning framework uses a centralized training with decentralized execution (CTDE) paradigm to jointly train the system coordination agent, the DC engineering agent, and the energy storage agent. During the training phase, global information is utilized to improve policy learning stability and multi-agent cooperation. During the execution phase, real-time control is achieved solely through local observations and boundary coordination by each agent. This step uses the aforementioned candidate node selection results and capacity optimization results as shared prior inputs, forming a closed-loop feedback with the subsequent constraint satisfaction layer.

[0096] During the intensive training phase, all agents share the global value evaluator, and the global value function is denoted as... ,in, Represents the global state; The global value function represents the joint action composed of all agent actions, and learns the mapping relationship between the cross-sectional safety objective and multi-resource cooperative actions; through the global state With joint actions A unified assessment is conducted to identify the comprehensive impact of each local action on the overall cross-sectional safety target; in the decentralized execution phase, the first... Each agent relies solely on its own local observations Output Action During the execution phase, each agent independently outputs control actions based on local observations to meet the real-time requirements of online control. These local observations include sensitivity characteristics and capacity boundary information. During training, a multi-resource collaborative control strategy is learned through a combination of centralized training and distributed execution. Counterfactual baselines are used to reduce reward attribution bias, establishing a stable mapping between global grid section safety targets and local control actions. The system coordination agent is responsible for generating a global coordination boundary based on the target grid section risk level, power flow sensitivity absolute value ranking, and energy storage capacity configuration. The DC engineering agent learns coordinated modulation strategies among multiple DC lines under the constraints of the global coordination boundary, while the energy storage agent learns local rapid charging and discharging strategies under the same constraints, thus balancing global coordination and real-time response. Therefore, the system coordination agent can learn resource boundary allocation logic, while local agents can learn the optimal response mode under the combined constraints of sensitivity priors and capacity boundaries. After training convergence, the system can output collaborative control strategies that meet section safety requirements under different operating modes and different renewable energy disturbance scenarios, thereby achieving dynamic coordination between multi-DC power modulation, energy storage charging and discharging, and renewable energy regulation.

[0097] Specifically, the target grid section risk level is obtained based on the real-time load rate, forward and reverse power flow exceedance levels, power flow safety margin, and bottleneck severity indicators of the monitored section. The section risk level serves as the state input for the system coordination agent to generate the global coordination boundary. The comprehensive impact of each local action on the overall section safety target is used to evaluate the improvement in section safety margin and future cumulative rewards after executing joint actions under the section risk level. The global value estimator uses the global state and joint actions as inputs, combined with bottleneck severity indicators, safety constraint satisfaction indicators, energy storage investment costs, and the projection correction of the original control actions, to calculate the cumulative return estimate corresponding to the joint actions. This allows for the assessment and identification of the comprehensive impact of each local action on the overall section safety target.

[0098] Introducing candidate node sensitivity indices and corresponding capacity configuration results into the local states of the energy storage agent characterizes the support capability and available adjustment range of each energy storage node for the target section, ensuring that energy storage actions are simultaneously constrained by local state of charge, node support capability, and capacity boundaries. This forms a hierarchical collaborative control mechanism where the system coordination agent is responsible for generating the global boundary, and the DC engineering agent and energy storage agent are responsible for executing local actions, thereby reducing the complexity of directly solving high-dimensional joint control problems.

[0099] To reduce reward attribution bias during multi-agent training, a counterfactual baseline mechanism is introduced. This mechanism measures the marginal contribution of the current agent's action to the global control effect by constructing a comparison method where "other agents' actions remain unchanged, only the current agent's action is replaced." The advantage function of each agent is:

[0100] ;

[0101] In the formula, Indicates the first Each agent in the global state Joint actions The dominant function under these conditions; Indicates except the first The set of actions of the other intelligent agents outside the intelligent body; Indicates the first The baseline action or reference action of an agent.

[0102] The above expression quantifies the difference in global value between the two scenarios: "using actual actions" and "using baseline actions." The marginal contribution of each agent's actions to the overall control effect is analyzed, thereby improving the accuracy of policy update direction and training convergence.

[0103] Furthermore, the constraint satisfaction layer maps the learned original control actions to a feasible domain that satisfies engineering constraints. This layer does not directly replace the reinforcement learning strategy, but rather, while preserving the policy's control intent, it performs feasibility modifications on the original control actions and feeds back the projection correction amount and active constraint information to the online update process, thus forming a closed loop of "policy generation - safety correction - model correction".

[0104] The constraint satisfaction layer uses the original control actions As input, construct a feasible region that satisfies power balance constraints, equipment output boundary constraints, equipment ramp-up constraints, and energy storage operation constraints. The original control action is then projected onto the nearest feasible control action point within the feasible region. The upper and lower bounds of the energy storage output in the feasible region are dynamically determined by the energy storage configuration capacity results obtained in step S3, thereby allowing the planning layer capacity results to directly constrain the operation layer action space.

[0105] Specifically, the power balance constraint is:

[0106] ;

[0107] In the formula, Represents the power balance coefficient matrix; This represents the right-hand side of the power balance constraint. This constraint is used to ensure that the overall power regulation of the system meets the real-time balance requirements under the executed action.

[0108] Specifically, the equipment ramp-up constraints are as follows:

[0109] ;

[0110] In the formula, Indicates device The action performed in the previous control moment; Indicates device The upper limit of the climbing rate; This indicates the control time step. This constraint is used to prevent the control command from changing too quickly, avoiding equipment operation shocks and control oscillations.

[0111] Specifically, the equipment output boundary constraints are as follows:

[0112] ;

[0113] In the formula, and Indicates device The minimum and maximum permissible action values. This constraint is used to ensure that the actions performed do not exceed the physical output capabilities of the DC power plant, new energy source, and energy storage unit.

[0114] Specifically, the constraints for energy storage operation are:

[0115] ;

[0116] In the formula, Indicates action After execution The state of charge of each energy storage unit; and This indicates the lower and upper limits of permissible operation for energy storage. This constraint is used to prevent energy storage units from overcharging, over-discharging, and becoming unsustainable.

[0117] The original control action is mapped to the feasible region that satisfies the constraints through orthogonal projection, and the feasible control action is output:

[0118] ;

[0119] ;

[0120] In the formula, Indicates feasible regulatory actions; This represents the feasible region that satisfies power balance constraints, equipment output boundary constraints, equipment ramp-up constraints, and energy storage operation constraints. This represents a combined action consisting of the actions of all agents. Indicates the original regulatory action; This represents the power vector of each DC project; Represents the new energy regulation power vector; This represents the energy storage charging and discharging power vector.

[0121] The purpose of the aforementioned projection model is to ensure that the executed actions meet engineering safety constraints while preserving the original intent of the learning strategy as much as possible, thereby effectively connecting the output of the reinforcement learning strategy with the actual executable control commands of the power grid. Furthermore, the projection offset, the active set of constraints, and the occupancy of the feasible region boundaries can be written back to the global value estimator or the online update module as additional feedback quantities to correct subsequent policy parameters and reward estimates.

[0122] The constraint satisfaction layer receives the original action vector output by the three-layer MARL framework in each control cycle. The action is then projected and corrected based on the aforementioned feasible region definition to obtain an execution action that satisfies engineering constraints. The core function of this layer is to transform the strategy output from a "theoretically optimal action" into an "engineerably executable action," while simultaneously feeding back the projection correction amount, active constraint labels, and feasible region boundary occupancy information to the online update stage for revising subsequent strategy parameters. By performing minimum offset projection on the original action, the constraint satisfaction layer can eliminate constraint violation risks while preserving the strategy's control intent as much as possible, improving the method's engineering feasibility and reducing control performance losses caused by action correction. Since the energy storage action boundary is dynamically determined by the capacity result obtained in step S3, this projection process also constitutes a direct coupling interface between the planning layer's capacity configuration and the operation layer's real-time control.

[0123] Furthermore, online model updates are used to maintain the adaptive capability of the strategy during long-term operation, enabling the model to remain effective under conditions of changes in operating mode, changes in the volatility characteristics of new energy sources, and changes in equipment status. To maintain consistency with the near-end strategy optimization algorithm used in step S3, online updates only use trajectory samples collected within the most recent scheduling cycle of the current strategy or its adjacent versions for fine-tuning within the same strategy.

[0124] During online operation, the system caches the state, actions, rewards, and projection correction results from one or more recent scheduling cycles using a sliding time window. Trajectory batches are constructed chronologically, and when update trigger conditions are met, the advantage function and policy pruning objective are calculated to incrementally update the parameters of the three-layer multi-agent reinforcement learning framework. The trajectory samples prioritize representative cross-sectional limit-crossing and constraint-active scenarios under the current operating mode to improve the policy's adaptability to critical conditions. The policy pruning objective is expressed as:

[0125] ;

[0126] In the formula, This indicates that the current policy and the sampling policy are in... The probability ratio at time t; Indicates the current policy network parameters; This represents the advantage function estimated from trajectory samples, which consist of the state, action, reward, and the projection correction of the original control action within the scheduling period. This indicates the pruning threshold for the strategy.

[0127] Specifically, the update trigger conditions include at least one of the following: the number of trajectory samples within the most recent sliding time window reaches a preset sample threshold; the load rate of any monitoring section exceeds a preset risk threshold; the projection correction of the original control action exceeds a preset correction threshold; the distribution of new energy output prediction errors or the power grid operation mode changes beyond a preset threshold; or a preset periodic update time is reached. The dominance function in the online update uses the aforementioned counterfactual dominance function, combined with the reward sequence and value function of the most recent scheduling cycle for estimation.

[0128] During online operation, newly acquired trajectories participate in updates in the form of sliding time windows, and the model parameters are fine-tuned with small step increments when the trigger conditions are met. This improves the method's adaptability to changes in operating mode, distribution of new energy output, and equipment operating conditions, while avoiding the introduction of long-term experience replay mechanisms that conflict with optimization of the same strategy.

[0129] After offline training, the MARL framework is deployed in the real-time decision support system of the target power grid dispatch and control center, executing the coordinated control strategy according to the principle of "the system coordination layer providing global boundaries and the local execution layer completing rapid responses." During online execution, the system coordination layer outputs resource boundary information based on section risk, sensitivity ranking results, and energy storage capacity configuration results. Under these boundary constraints, the local execution layer independently generates control actions based on local observations, thereby achieving a unity of global coordination and local rapid response. During online operation, the system continuously collects new state transition samples and performs incremental updates based on trajectory samples generated by the current strategy or adjacent strategy versions within the most recent scheduling cycle. The updated samples simultaneously include the projection correction returned by the constraint satisfaction layer and active constraint information, thus maintaining the model's adaptability to changes in operating modes, changes in the characteristics of renewable energy fluctuations, and changes in equipment status.

[0130] This invention takes a provincial power system as the verification object. The system includes multiple 500 kV hub substations and several high-voltage direct current (HVDC) projects. The installed capacity of new energy power generation accounts for more than 30% of the total installed capacity of the system. On a typical operating day, the photovoltaic output accounts for 25% of the real-time load of the system. The power flow standard deviation of key transmission sections exceeds ±20%, and the section bottleneck problem is prominent, which is highly representative.

[0131] See Figures 3 to 5 The present invention compares the simulation verification results of the proposed method with those of the baseline method. Detailed performance index data are shown in the table below:

[0132]

[0133] Simulation results show that, compared with the existing rule-based multi-DC coordinated control method (baseline method A), the method of the present invention reduces the standard deviation of power flow at key sections by about 32.3%, reduces the frequency of section overruns by about 45.2%, and increases the utilization rate of energy storage resources by about 27.5 percentage points. It has superior performance and good engineering applicability in the task of eliminating bottlenecks in high-proportion power electronic grid sections.

[0134] The above experimental results verify the comprehensive superiority of the method proposed in this invention in terms of cross-sectional bottleneck elimination effect, safety constraint guarantee, and energy storage resource utilization rate. Figure 6 As shown, the power flow sensitivity heatmap illustrates the influence of each node on the target cross-section, used to identify key nodes that significantly impact cross-section safety. A higher sensitivity value indicates a more significant power regulation effect of that node on suppressing power flow at the target cross-section. Figure 7 As shown, the comparative analysis of energy storage optimization configuration results shows the difference in cross-sectional support capability between the two schemes of "prioritizing capacity allocation to high-sensitivity nodes" and "balanced capacity allocation to all candidate nodes". This further illustrates that prioritizing capacity deployment to high-sensitivity nodes and continuing to pass this sensitivity prior to subsequent collaborative control and action projection can achieve better cross-sectional support effect and more stable online regulation performance under the same total capacity conditions.

[0135] See Figure 8 The present invention also provides a grid cross-section bottleneck elimination system for multi-agent and energy storage collaborative optimization, characterized in that the system is used to implement the above-mentioned grid cross-section bottleneck elimination method for multi-agent and energy storage collaborative optimization, and the system includes:

[0136] The three-layer MARL framework construction module is used to collect real-time operation data of the target power grid and construct a three-layer multi-agent reinforcement learning framework including a system coordination agent, a DC engineering agent, and an energy storage agent. The system coordination agent is used to characterize global operation information, the DC engineering agent is used to characterize the local adjustability and operation constraints of the DC engineering, and the energy storage agent is used to characterize the local adjustability and operation constraints of the energy storage unit.

[0137] The candidate node set acquisition module is used to calculate the power flow sensitivity matrix of the target power grid section set to each node, and to filter the energy storage candidate nodes based on the power flow sensitivity matrix to obtain the candidate node set.

[0138] The energy storage configuration capacity result acquisition module is used to perform collaborative optimization of the energy storage configuration capacity of each candidate node in a set of uncertain new energy output scenarios using a near-end strategy optimization algorithm to obtain the energy storage configuration capacity result.

[0139] The feasible control action acquisition module is used to train the three-layer multi-agent reinforcement learning framework to output the original control actions, and to map the original control actions to the feasible domain that meets the constraints through orthogonal projection, and output the feasible control actions.

[0140] The specific functions and implementation steps of the three-layer MARL framework construction module, candidate node set acquisition module, energy storage configuration capacity result acquisition module, and feasible control action acquisition module are described above and will not be repeated here. Through the coordinated cooperation of the above modules, a complete closed-loop functional chain can be realized, from state awareness, candidate node screening, capacity optimization, hierarchical collaborative control to safe execution. Among them, candidate screening results, capacity configuration results, and projection feedback information are continuously transmitted between modules and work together to influence the final control decision.

[0141] See Figure 9 The present invention also provides a grid section bottleneck elimination device for multi-agent and energy storage collaborative optimization, including a memory and a processor;

[0142] The memory is used to store computer program code and transmit the computer program code to the processor;

[0143] The processor is configured to execute, according to instructions in the computer program code, the grid cross-section bottleneck elimination method described above for multi-agent and energy storage collaborative optimization.

[0144] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the grid cross-section bottleneck elimination method for multi-agent and energy storage collaborative optimization as described above.

[0145] Generally, the computer instructions for implementing the method of the present invention can be carried on any combination of one or more computer-readable storage media. Non-transitory computer-readable storage media can include any computer-readable medium except for the signal itself, which is temporarily propagating.

[0146] Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EKROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0147] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. In particular, Python, suitable for neural network computation, and platform frameworks such as TensorFlow and PyTorch can be used. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer or to an external computer (e.g., via the Internet using an Internet service provider) through any type of network, including a local area network (LAN) or a wide area network (WAN).

[0148] The aforementioned equipment and non-transitory computer-readable storage media can be found in the detailed description of a method for eliminating grid cross-section bottlenecks through multi-agent and energy storage collaborative optimization and its beneficial effects, which will not be repeated here.

[0149] Although embodiments of the present invention have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for eliminating grid cross-section bottlenecks through multi-agent and energy storage collaborative optimization, characterized in that, include: Real-time operation data of the target power grid is collected, and a three-layer multi-agent reinforcement learning framework is constructed, which includes a system coordination agent, a DC engineering agent, and an energy storage agent. The system coordination agent is used to characterize global operation information, the DC engineering agent is used to characterize the local adjustability and operation constraints of the DC engineering, and the energy storage agent is used to characterize the local adjustability and operation constraints of the energy storage unit. Calculate the power flow sensitivity matrix of the target power grid section set to each node for injected power, and filter the energy storage candidate nodes based on the power flow sensitivity matrix to obtain the candidate node set; A near-end strategy optimization algorithm is used to collaboratively optimize the energy storage configuration capacity of each candidate node in a set of uncertain new energy output scenarios to obtain the energy storage configuration capacity result; The three-layer multi-agent reinforcement learning framework is trained to output the original control actions, and the original control actions are mapped to the feasible domain that satisfies the constraints through orthogonal projection, and the feasible control actions are output.

2. The method for eliminating grid cross-section bottlenecks through multi-agent and energy storage collaborative optimization according to claim 1, characterized in that, Current sensitivity is: ； In the formula, Represents a node The injected active power on the cross section The sensitivity of the active power flow; Indicates cross-section The corresponding branch aggregation coefficient vector; This represents the diagonal matrix formed by the branch reactances; Represents the branch-node correlation matrix; This represents the nodal susceptance matrix after deducting the equilibrium point; Represents a node The unit injection vector; This represents the reference vector corresponding to the balancing node.

3. The method for eliminating grid cross-section bottlenecks through multi-agent and energy storage collaborative optimization according to claim 2, characterized in that, The process of filtering candidate energy storage nodes based on the power flow sensitivity matrix to obtain a candidate node set includes: For each cross section, the absolute value of the tidal current sensitivity is used. Sort the nodes from largest to smallest, and use the cumulative coverage criterion to determine the number of candidate nodes for each section. Then, take the union of the candidate nodes for each section to form the final candidate node set. ; The number of candidate nodes for each cross section is: ； In the formula, Indicates cross-section The number of candidate nodes selected; Indicates by The th in descending order Index of each node; This indicates the preset cumulative sensitivity coverage threshold. .

4. The method for eliminating grid cross-section bottlenecks through multi-agent and energy storage collaborative optimization according to claim 1, characterized in that, The reward function of the near-end policy optimization algorithm is: ； ； In the formula, express The reward value at any given moment; Indicates cross-section The severity index of the checkpoint; Indicates the first Energy storage configuration capacity corresponding to each candidate node; This represents the equivalent investment cost per unit capacity of energy storage. This indicates that the safety constraints satisfy the indicated quantity; , , These are the weighting factors, used to balance the safety of the cross section, investment costs, and the degree to which feasibility constraints are met. Indicates cross-section Positive load rate, , Indicates cross-section Real-time active power flow, Indicates cross-section The permissible limits under positive current; Indicates cross-section Reverse load rate, , Indicates cross-section Permissible limits under reverse current.

5. The method for eliminating grid cross-section bottlenecks through multi-agent and energy storage collaborative optimization according to claim 1, characterized in that, The training of the three-layer multi-agent reinforcement learning framework outputs the original control actions, including: During the intensive training phase, all agents share the global value evaluator, and the global value function is denoted as... ,in, Represents the global state; This represents a joint action consisting of the actions of all agents, expressed through the global state. With joint actions A unified assessment is conducted to identify the comprehensive impact of each local action on the overall cross-sectional safety target; During the distributed execution phase, the system coordination agent generates a global coordination boundary based on the target grid section risk level, the ranking results of the absolute value of power flow sensitivity, and the energy storage configuration capacity results. Under the constraints of the global coordination boundary, the DC engineering agent learns the coordination modulation strategy between multiple DC circuits, and the energy storage agent learns the local fast charging and discharging strategy under the constraints of the global coordination boundary.

6. The method for eliminating grid cross-section bottlenecks through multi-agent and energy storage collaborative optimization according to claim 5, characterized in that, During the training of the three-layer multi-agent reinforcement learning framework, the first... The advantage function of each agent is: ； In the formula, Indicates the first Each agent in the global state Joint actions The dominant function under these conditions; Indicates except the first The set of actions of the other intelligent agents outside the intelligent body; Indicates the first The baseline action or reference action of an agent.

7. The grid section bottleneck elimination method for multi-agent and energy storage collaborative optimization according to claim 5, characterized in that, The step of mapping the original control action to a feasible domain that satisfies the constraints through orthogonal projection and outputting a feasible control action includes: ； ； In the formula, Indicates feasible regulatory actions; This represents the feasible region that satisfies power balance constraints, equipment output boundary constraints, equipment ramp-up constraints, and energy storage operation constraints. This represents a combined action consisting of the actions of all agents. Indicates the original regulatory action; This represents the power vector of each DC project regulation; Represents the new energy regulation power vector; This represents the energy storage charging and discharging power vector.

8. The method for eliminating grid cross-section bottlenecks through multi-agent and energy storage collaborative optimization according to claim 6, characterized in that, When the update trigger condition is met, the advantage function and policy pruning objective are calculated, and the parameters of the three-layer multi-agent reinforcement learning framework are incrementally updated. The policy pruning objective is expressed as: ； In the formula, This indicates that the current policy and the sampling policy are in... The probability ratio at time t; Indicates the current policy network parameters; This represents the advantage function estimated from trajectory samples, which consist of the state, action, reward, and the projection correction of the original control action within the scheduling period. This indicates the pruning threshold for the strategy.

9. A grid cross-section bottleneck elimination system for multi-agent and energy storage collaborative optimization, characterized in that, The system is used to implement the method according to any one of claims 1 to 8, the system comprising: The three-layer MARL framework construction module is used to collect real-time operation data of the target power grid and construct a three-layer multi-agent reinforcement learning framework including a system coordination agent, a DC engineering agent, and an energy storage agent. The system coordination agent is used to characterize global operation information, the DC engineering agent is used to characterize the local adjustability and operation constraints of the DC engineering, and the energy storage agent is used to characterize the local adjustability and operation constraints of the energy storage unit. The candidate node set acquisition module is used to calculate the power flow sensitivity matrix of the target power grid section set to each node, and to filter the energy storage candidate nodes based on the power flow sensitivity matrix to obtain the candidate node set. The energy storage configuration capacity result acquisition module is used to perform collaborative optimization of the energy storage configuration capacity of each candidate node in a set of uncertain new energy output scenarios using a near-end strategy optimization algorithm to obtain the energy storage configuration capacity result. The feasible control action acquisition module is used to train the three-layer multi-agent reinforcement learning framework to output the original control actions, and to map the original control actions to the feasible domain that meets the constraints through orthogonal projection, and output the feasible control actions.

10. A grid cross-section bottleneck elimination device for multi-agent and energy storage collaborative optimization, characterized in that, Including memory and processor; The memory is used to store computer program code and transmit the computer program code to the processor; The processor is configured to execute the method as described in any one of claims 1 to 8 according to instructions in the computer program code.

Images