Virtual power plant collaborative control method based on multi-agent hierarchical reinforcement learning

Abstract

The application discloses a virtual power plant cooperative control method based on multi-agent layered reinforcement learning, and a layered multi-agent system (MAS) processes complex tasks through hierarchical cooperation and distributed decision among agents, and specifically comprises the following steps: S1, an attribute information of a single agent is acquired by a bottom execution layer, specific equipment is directly controlled, and an inverter output power is adjusted or a load transfer instruction is executed according to a middle management layer; S2, a middle management layer receives an instruction of a top coordination layer, a sub-group is mixedly divided according to a region or a resource type, the instruction of the top coordination layer is executed, and real-time data is fed back; S3, the top coordination layer serves as a global decision center, is responsible for power market strategy formulation and resource aggregation optimization, and finally outputs a structured decision instruction and dynamic feedback; after the virtual power plant (VPP) receives the structured decision instruction and dynamic feedback output by the top coordination layer, layered processing and dynamic optimization are carried out. The application can obviously reduce delay and reduce load.

Description

Technical Field

[0001] This invention relates to the field of electrical technology, and in particular to a collaborative control method for virtual power plants based on multi-agent hierarchical reinforcement learning. Background Technology

[0002] The mainstream control architecture of current virtual power plants (VPPs) generally adopts a centralized energy management system (EMS). The core problem is the disaster of computational dimension. When the number of distributed resources exceeds 5,000, the power flow calculation time reaches 500ms. According to power grid fault statistics, the risk of single-point failure is high, and a single-point failure can easily lead to the paralysis of the entire network. According to statistics from previous years, 87% of large-scale power outages are due to communication interruptions in the control center. Fixed parameter models are difficult to adapt to the fluctuations in the output of new energy sources. Actual measurements of a provincial power grid show that photovoltaic prediction errors lead to a voltage over-limit rate of 12.6%.

[0003] Therefore, a collaborative control method for virtual power plants based on multi-agent hierarchical reinforcement learning is proposed to solve the above problems. Summary of the Invention

[0004] The purpose of this invention is to provide a collaborative control method for virtual power plants based on multi-agent hierarchical reinforcement learning, which reduces local decision latency to milliseconds, enables distributed autonomy, automatic isolation of faulty nodes, lowers communication load, and achieves adaptive local agent adjustment for new energy.

[0005] To achieve the above objectives, this invention provides a collaborative control method for virtual power plants based on hierarchical reinforcement learning of multiple agents. The virtual power plant (VPP) completes the collaborative control of distributed resources through a hierarchical multi-agent system (MAS). The MAS processes complex tasks through hierarchical cooperation and distributed decision-making among the agents, specifically including a top-level coordination layer, a middle management layer, and a bottom-level execution layer, and includes the following steps: Step S1: The bottom execution layer obtains the attribute information of a single intelligent agent, directly controls the specific device, and adjusts the inverter output power or executes load transfer commands according to the middle management layer. Step S2: The middle management layer receives instructions from the top coordination layer, divides the data into subgroups according to region or resource type, executes the instructions from the top coordination layer, and feeds back real-time data. Step S3: The top-level coordination layer, as the global decision-making center, is responsible for formulating power market strategies and optimizing resource aggregation, and ultimately outputs structured decision instructions and dynamic feedback; After receiving structured decision instructions and dynamic feedback from the top-level coordination layer, the Virtual Power Plant (VPP) performs layered processing and dynamic optimization.

[0006] Preferably, in step S1, the attribute information of a single intelligent agent includes the remaining power and location information of the intelligent agent, as well as the energy type; the underlying execution layer interacts directly with the physical environment, specifically including physical device driving, executing inverter power adjustment and energy storage charging and discharging control commands, wherein the photovoltaic inverter adjusts its output power according to changes in sunlight, with a response time of ≤110ms, and the energy storage charging and discharging control accuracy reaches ±1% of SOC.

[0007] Preferably, in step S2, the intermediate management layer divides regions based on the power grid topology and geographical boundaries, dividing them into distribution subgroups according to resources within the same transformer's power supply range, voltage level subgroups according to resources connected to the same voltage level, and administrative subgroups according to the dispatch jurisdiction of the region or power grid; and classifies resources by type according to focusing on technical characteristics and performance, specifically including flexible power generation subgroups, fluctuating power supply subgroups, adjustable load subgroups, and intelligent collaborative subgroups, while deploying edge computing gateways for each subgroup.

[0008] Preferably, in step S2, executing the instructions of the top-level coordination layer and feeding back real-time data specifically includes the following steps: S21: Receive instructions, parse instruction requirements, and match resource tags; S22: Employs LoRa+5G dual-channel redundant transmission commands to achieve dynamic resource scheduling; S23: Real-time control, data feedback, and tiered data acquisition for different data types. Defines the golden signals for key performance indicators.

[0009] Preferably, in step S3, the execution mechanism of the cross-resource optimization model is used to complete the multi-resource collaborative scheduling, and the objective function of the core optimization model is expressed as: ; In the formula, Indicates current market returns. This represents the current market price of electricity during time period t; This represents the winning bid volume for the current market during time period t. Indicates real-time market returns. This represents the real-time market price of electricity during time period t. This represents the winning bid volume in the real-time market during time period t. This indicates revenue from FM service. This represents the price of frequency modulation ancillary services during time period t. This represents the frequency regulation reserve capacity during time period t; Indicates operating costs, Indicates energy storage loss; The constraints are: Dynamic balance refers to the real-time balance between power generation and consumption. Representing resources i exist t Power processing for a given time period: a positive result indicates power generation or discharge, while a negative result indicates power charging or consumption. express t Rigid load demand during specific time periods; This represents the minimum technical processing required for resource i. Representing resources i Maximum technical output; The climbing constraint is specifically manifested as a limitation on the rate of adjustment. Representing resources i Maximum gradeability; Energy storage dynamics are represented as state of charge updates. Representing resources i exist t The status of energy storage during a given time period Indicates charging efficiency. Indicates discharge efficiency; express t Charging power during the period express t Discharge power over a given period of time; Power grid security is represented as node voltage constraints; This represents the per-unit value of the voltage deviation at the virtual power plant's grid connection point. This indicates the maximum permissible safe voltage deviation threshold of the power grid; The operating cost function for adjustable load is expressed as: = ; in, This represents the unit compensation cost for load reduction. This represents the linear cost factor for increased load. This represents the secondary cost coefficient for increased load. Indicates time; The energy storage loss function is expressed as: = ) 0.8 + ( -0.5) 2 ; in, The cyclic decay coefficient is... To deviate from the optimal SOC penalty coefficient, Indicates energy storage i The rated capacity, with an exponent of 0.8 representing the non-linearity factor of the attenuation curve.

[0010] Preferably, the Virtual Power Plant (VPP) includes a resource access and aggregation module, a core control module, a market transaction module, an operation monitoring module, and an interaction module; The resource access and aggregation module includes the access of distributed power sources and the management of flexible loads. Distributed power sources are used to integrate photovoltaic, wind power, energy storage or other distributed energy sources and support the IEEE 1547 / IEC 61850 communication protocol. The management of flexible loads includes accessing adjustable loads, including air conditioning clusters, charging piles or industrial production lines, and adjusting the load curve through demand response protocols.

[0011] Preferably, the core control module includes a collaborative optimization engine and a prediction subsystem. The collaborative optimization engine optimizes the scheduling strategy in real time based on electricity price signals, grid constraints and resource characteristics, and uses linear programming algorithm and reinforcement learning algorithm to control the response delay within 300ms. The forecasting subsystem uses an LSTM model to forecast renewable energy output with an accuracy greater than 93%. It also forecasts electricity prices and load demand based on real-time electricity market prices.

[0012] Preferably, the market trading module includes spot market bidding and ancillary services market. Spot market bidding aggregates resources to participate in electricity bidding transactions to maximize arbitrage profits. The ancillary services market provides frequency regulation AGC and reserve capacity assistance, with a frequency regulation response speed of less than 3 seconds.

[0013] Preferably, the operation monitoring module includes a digital twin platform and an anomaly diagnosis system. The digital twin platform constructs a power grid-based indoor mapping to simulate extreme scenarios such as typhoons or faults, and outputs emergency plans and safety boundaries. The anomaly diagnosis system uses time-series data analysis and graph neural networks (GNNs) to locate equipment faults using a knowledge graph.

[0014] Preferably, the interaction module includes a terminal control interface that sends charging / discharging commands or start / stop commands to the owner of the resource.

[0015] Therefore, the virtual power plant cooperative control method based on multi-agent hierarchical reinforcement learning with the above-described structure has the following beneficial effects: (1) In this invention, the hierarchical MAS constructs a three-level intelligent agent network of “global optimization-regional coordination-local autonomy”, while maintaining the advantages of centralized scheduling, and obtains the elasticity and efficiency of distributed systems, so that VPP evolves from “passive response type” to “active evolution type” energy hub.

[0016] (2) This invention realizes distributed resource collaborative control through MAS, which improves the elasticity and robustness of the system. Specifically, it is manifested in single-point fault isolation: local intelligent agent failure does not affect the overall system, dynamic topology reconstruction: local autonomous mode is triggered when communication is interrupted, and fault recovery speed: shortened from minutes to seconds in the traditional centralized system.

[0017] (3) The present invention enhances the anti-disturbance capability and improves the renewable energy fluctuation absorption capability by 40%-60%, while the voltage flicker suppression rate of centralized control is improved by 75%.

[0018] (4) The present invention reduces the communication bandwidth requirement by 65% ​​and the computing load of the central node by 80% when transmitting only optimized boundary conditions.

[0019] The method of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0020] Figure 1 This is a flowchart of the hierarchical processing and dynamic optimization process in this invention; Figure 2 This is a flowchart of the objective function model solution in an embodiment of the present invention; Figure 3 This is the top-level coordination layer instruction execution mechanism in this embodiment of the invention. Detailed Implementation

[0021] The method of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0022] Unless otherwise defined, the methodological or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0023] The terms "comprising" or "including" as used in this invention mean that the element preceding the term encompasses the element listed after the term, and do not exclude the possibility of encompassing other elements as well. Terms such as "inner," "outer," "upper," and "lower" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0024] Example This invention provides a collaborative control method for virtual power plants based on hierarchical reinforcement learning of multiple agents. The virtual power plant (VPP) completes the collaborative control of distributed resources through a hierarchical multi-agent system (MAS). The MAS processes complex tasks through hierarchical cooperation and distributed decision-making among the agents, specifically including a top-level coordination layer, a middle management layer, and a bottom-level execution layer. The method includes the following steps: like Figure 1 As shown, the specific process of hierarchical processing and dynamic optimization is as follows: The top-level coordination layer, acting as the global decision-making center, outputs structured decision commands and dynamic feedback. The middle management layer, upon receiving these structured commands, divides the data into subgroups based on region or resource type, completing initial command parsing and label matching. Simultaneously, it aggregates the data from each subgroup, summarizing the resource operating status within the region. Using an edge calculator, it decomposes the resource operating status within the region into commands and distributes them to the bottom-level execution layer. The bottom-level execution layer obtains the attribute information of individual intelligent agents and directly controls specific equipment. During execution, it collects equipment operating data in real time and feeds it back to the middle management layer. Subsequently, the middle management layer performs deviation analysis on the feedback data, identifying deviations between the actual operating status and command requirements. Based on the deviation results, it performs dynamic calibration and adjusts the subgroup control commands. Finally, the virtual power plant aggregates the data that meets the deviation analysis criteria, optimizes the operating results, and feeds back the closed-loop to the top-level coordination layer, forming a complete collaborative control loop and achieving efficient linkage and continuous optimization across all levels.

[0025] Step S1: The bottom execution layer obtains the attribute information of a single intelligent agent, directly controls the specific device, and adjusts the inverter output power or executes load transfer commands according to the middle management layer. In step S1, the attribute information of a single intelligent agent includes the remaining power and location information of the intelligent agent, as well as the energy type; the bottom execution layer interacts directly with the physical environment, specifically including physical device driving, executing inverter power adjustment and energy storage charging and discharging control commands, wherein the photovoltaic inverter adjusts the output power according to the change of light, with a response time of ≤110ms, and the energy storage charging and discharging control accuracy reaches ±1% of SOC.

[0026] Step S2: The middle management layer receives instructions from the top coordination layer, divides the subgroups according to region or resource type, executes the instructions from the top coordination layer, and feeds back real-time data; Verification of the division effect: Actual measured data of 15MW VPP are shown in Table 1; Table 1

[0027] In step S2, the intermediate management layer divides the region based on the power grid topology and geographical boundaries. It divides the region into distribution subgroups based on resources within the same transformer power supply range (e.g., electrical distance ≤ 1km), into voltage level subgroups based on resources connected to the same voltage level (voltage is 10kV or 380V), and into administrative subgroups based on the dispatch jurisdiction of the region or power grid. Based on the focus on technical characteristics and performance, the resource types are divided as shown in Table 2, specifically including flexible power generation subgroups, fluctuating power supply subgroups, adjustable load subgroups, and intelligent collaborative subgroups. At the same time, edge computing gateways are deployed in each subgroup to reduce cross-subgroup latency communication.

[0028] Table 2

[0029] like Figure 3 As shown, in step S2, executing the instructions of the top-level coordination layer and feeding back real-time data specifically includes the following steps: S21: Receive instructions, parse instruction requirements, and match resource tags; S22: Employs LoRa+5G dual-channel redundant transmission commands, with dynamic switching latency <90ms, enabling dynamic resource scheduling; S23: Real-time control, data feedback, and tiered data acquisition for different data types. Defines the golden signals for key performance indicators.

[0030] Step S3: The top-level coordination layer, acting as the global decision-making center, is responsible for formulating electricity market strategies and optimizing resource aggregation, ultimately outputting structured decision instructions and dynamic feedback; the closed-loop process of instruction execution and data feedback is as follows: Figure 3 As shown: Top-level coordination layer -> Middle management layer: Issues dispatch instructions (e.g., 15:00 peak shaving of 8MW); Middle Management Layer -> Resource Tagging System: Parse instructions and match resource tags (Region = Industrial Zone A, Type = Adjustable Load); Middle management layer -> Edge controller: Decompose instructions to the equipment layer (air conditioning group power reduction 2.3MW + production line delayed start 5.7MW); Edge controller -> Terminal device: Execute specific control (air conditioning temperature +1℃, production line B starts after a 15-minute delay). Terminal device -> Edge controller: Feedback of real-time status (actual load reduction 2.1MW, SOC=65%). Edge controller -> Middle management layer: Aggregate subgroup data (total load reduction of 7.8MW in Industrial Zone A); Middle management layer -> Top coordination layer: Feedback on execution results (target achievement rate 97.5%, delay 42ms).

[0031] In step S3, the execution mechanism of the cross-resource optimization model is used to complete the multi-resource collaborative scheduling. The objective function of the core optimization model is expressed as: In the formula, Indicates current market returns. Indicates the current market t Electricity prices during specific time periods; It is the current market t The winning bid volume for the specified time period; Indicates real-time market returns. Indicates real-time market t Electricity prices during different time periods This represents the winning bid volume in the real-time market during time period t. This indicates revenue from FM service. Indicates frequency modulation ancillary service t Prices for different time periods This represents the frequency regulation reserve capacity during time period t; Indicates operating costs, Represents energy storage loss; the solution process of the objective function model. Figure 2 As shown, the solution process is as follows: First, input relevant basic data such as market electricity price, resource output, load demand, and grid constraints to initialize the objective function model and select a suitable linear programming solver. Based on the relevant basic data, construct the objective function and corresponding constraints. Then, use the linear programming solver to solve the objective function model. If the convergence condition is met during the solution process, output the optimal scheduling strategy and send it to the hierarchical multi-agent system (MAS) for execution. If the convergence condition is not met, adjust the convergence variables and return the constraints. During execution, collect system operation data in real time and update the model parameters with the operation data to achieve continuous iterative optimization of the objective function model, ensuring the rationality and adaptability of the scheduling strategy.

[0032] The constraints are: Dynamic balance refers to the real-time balance between power generation and consumption. Representing resources i exist t Power processing for a given time period: a positive result indicates power generation or discharge, while a negative result indicates power charging or consumption. express t Rigid load demand during specific time periods; This represents the minimum technical processing required for resource i. Representing resources i Maximum technical output; The climbing constraint is specifically manifested as a limitation on the rate of adjustment. Representing resources i Maximum gradeability; Energy storage dynamics are represented as state of charge updates. Representing resources i exist t The status of energy storage during a given time period Indicates charging efficiency. Indicates discharge efficiency; express t Charging power during the period express t Discharge power over a given period of time; Power grid security is represented as node voltage constraints; This represents the per-unit value of the voltage deviation at the virtual power plant's grid connection point. This indicates the maximum permissible safe voltage deviation threshold of the power grid; The operating cost function for adjustable load is expressed as: = ; in, This represents the unit compensation cost for load reduction. This represents the linear cost factor for increased load. This represents the secondary cost coefficient for increased load. Indicates time; The energy storage loss function is expressed as: = ) 0.8 + ( -0.5) 2 ; in, The cyclic decay coefficient is... To deviate from the optimal SOC penalty coefficient, Indicates energy storage i The rated capacity, with an exponent of 0.8 representing the non-linearity factor of the attenuation curve.

[0033] After receiving structured decision instructions and dynamic feedback from the top-level coordination layer, the Virtual Power Plant (VPP) performs layered processing and dynamic optimization.

[0034] A virtual power plant (VPP) includes a resource access and aggregation module, a core control module, a market trading module, an operation monitoring module, and an interaction module. The resource access and aggregation module includes the access of distributed power sources and the management of flexible loads. Distributed power sources are used to integrate photovoltaic, wind power, energy storage or other distributed energy sources and support the IEEE 1547 / IEC 61850 communication protocol. The management of flexible loads includes accessing adjustable loads, including air conditioning clusters, charging piles or industrial production lines, and adjusting the load curve through demand response protocols.

[0035] The core control module includes a collaborative optimization engine and a prediction subsystem. The collaborative optimization engine optimizes the scheduling strategy in real time based on electricity price signals, grid constraints and resource characteristics. It uses linear programming algorithm and reinforcement learning algorithm to control the response delay within 300ms. The forecasting subsystem uses an LSTM model to forecast renewable energy output with an accuracy greater than 93%. It also forecasts electricity prices and load demand based on real-time electricity market prices.

[0036] The market trading module includes spot market bidding and ancillary services market. Spot market bidding aggregates resources to participate in electricity bidding transactions to maximize arbitrage profits. The ancillary services market provides frequency regulation AGC and reserve capacity assistance, with a frequency regulation response speed of less than 3 seconds.

[0037] The operation monitoring module includes a digital twin platform and an anomaly diagnosis system. The digital twin platform constructs an indoor mapping of the power grid to simulate extreme scenarios such as typhoons or faults, outputting emergency plans and safety boundaries. The anomaly diagnosis system uses time-series data analysis and graph neural networks (GNNs) to locate equipment faults using a knowledge graph.

[0038] The interaction module includes a terminal control interface, which sends charging / discharging commands or start / stop commands to the resource owner.

[0039] Implementation Cases A 200MW VPP in East China participates in the electricity spot market and frequency regulation ancillary services market; Current market: Based on forecasted bidding: 42MWh of energy storage discharge (peak hours) + 28MW of load reduction; Real-time Marketplace: Photovoltaic output exceeded expectations by 15% → Energy storage charging was activated to absorb excess power; FM market: 7MW of frequency regulation backup was provided by utilizing industrial interruptible load clusters. The economic benefits are shown in Table 3. Table 3

[0040] Optimization results: Prediction bias compensation rate: 89% → 94%; Strategy adjustment response delay: 12s → 3.8s; Annual total revenue increased by 31% compared to the traditional scheduling model.

[0041] Therefore, the present invention adopts the above-mentioned virtual power plant collaborative control method based on multi-agent hierarchical reinforcement learning, which reduces the local decision delay to the millisecond level, achieves distributed autonomy, automatically isolates faulty nodes, and reduces communication load, thus completing the adaptive adjustment of local intelligent agents for new energy.

[0042] Finally, it should be noted that the above embodiments are only used to illustrate the method of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the method of the present invention, and these modifications or equivalent substitutions should not cause the modified method to deviate from the spirit and scope of the method of the present invention.

Claims

1. A virtual power plant cooperative control method based on multi-agent hierarchical reinforcement learning, characterized in that: The Virtual Power Plant (VPP) achieves coordinated control of distributed resources through a hierarchical multi-agent system (MAS). The MAS processes complex tasks through hierarchical collaboration and distributed decision-making among the agents, specifically comprising a top-level coordination layer, a middle management layer, and a bottom-level execution layer. The steps are as follows: Step S1: The bottom-level execution layer acquires the attribute information of individual agents, directly controls specific equipment, and adjusts inverter output power or executes load transfer commands based on the middle management layer's instructions; Step S2: The middle management layer receives commands from the top-level coordination layer, divides the data into subgroups according to region or resource type, executes the commands, and provides real-time data feedback; Step S3: The top-level coordination layer, as the global decision-making center, is responsible for formulating power market strategies and optimizing resource aggregation, ultimately outputting structured decision commands and dynamic feedback. After receiving structured decision instructions and dynamic feedback from the top-level coordination layer, the Virtual Power Plant (VPP) performs layered processing and dynamic optimization.

2. The virtual power plant cooperative control method based on multi-agent hierarchical reinforcement learning as described in claim 1, characterized in that: In step S1, the attribute information of a single intelligent agent includes the remaining power and location information of the intelligent agent, as well as the energy type; the bottom execution layer interacts directly with the physical environment, specifically including physical device driving, executing inverter power adjustment and energy storage charging and discharging control commands, wherein the photovoltaic inverter adjusts the output power according to the change of light, with a response time of ≤110ms, and the energy storage charging and discharging control accuracy reaches ±1% of SOC.

3. The virtual power plant cooperative control method based on multi-agent hierarchical reinforcement learning as described in claim 2, characterized in that: In step S2, the intermediate management layer divides regions based on the power grid topology and geographical boundaries. It divides the regions into distribution subgroups based on resources within the same transformer's power supply range, into voltage level subgroups based on resources connected to the same voltage level, and into administrative subgroups based on the dispatch jurisdiction of the region or power grid. Based on the focus on technical characteristics and performance, the resources are divided into flexible power generation subgroups, fluctuating power supply subgroups, adjustable load subgroups, and intelligent collaborative subgroups, with edge computing gateways deployed for each subgroup.

4. The virtual power plant cooperative control method based on multi-agent hierarchical reinforcement learning as described in claim 3, characterized in that: In step S2, executing the instructions of the top-level coordination layer and feeding back real-time data specifically includes the following steps: S21: Receive instructions, parse instruction requirements, and match resource tags; S22: Employs LoRa+5G dual-channel redundant transmission commands to achieve dynamic resource scheduling; S23: Real-time control, data feedback, hierarchical acquisition of different data types, and definition of key performance indicators and golden signals.

5. The virtual power plant cooperative control method based on multi-agent hierarchical reinforcement learning as described in claim 4, characterized in that: In step S3, the execution mechanism of the cross-resource optimization model is used to complete the multi-resource collaborative scheduling. The objective function of the core optimization model is expressed as: ； In the formula, Indicates current market returns. Indicates the current market t Electricity prices during specific time periods; It is the current market t The winning bid volume for the specified time period; Indicates real-time market returns. Indicates real-time market t Electricity prices during different time periods Indicates real-time market t The winning bid volume for the specified time period This indicates revenue from FM service. Indicates frequency modulation ancillary service t Prices for different time periods express t Frequency regulation reserve capacity for a given time period; Indicates operating costs, Indicates energy storage loss; The constraints are: Dynamic balance refers to the real-time balance between power generation and consumption. Representing resources i exist t Power processing for a given time period: a positive result indicates power generation or discharge, while a negative result indicates power charging or consumption. express t Rigid load demand during specific time periods; Representing resources i Minimal technical processing, Representing resources i Maximum technical output; The climbing constraint is specifically manifested as a limitation on the rate of adjustment. Representing resources i Maximum gradeability; Energy storage dynamics are represented as state of charge updates. Representing resources i exist t The status of energy storage during a given time period Indicates charging efficiency. Indicates discharge efficiency; express t Charging power during the period express t Discharge power over a given period of time; Power grid security is represented as node voltage constraints; This represents the per-unit value of the voltage deviation at the virtual power plant's grid connection point. This indicates the maximum permissible safe voltage deviation threshold of the power grid; The operating cost function for adjustable load is expressed as: = ； in, This represents the unit compensation cost for load reduction. This represents the linear cost factor for increased load. This represents the secondary cost coefficient for increased load. Indicates time; The energy storage loss function is expressed as: = ) 0.8 + ( -0.5) 2 ； in, The cyclic decay coefficient is... To deviate from the optimal SOC penalty coefficient, Indicates energy storage i The rated capacity, with an exponent of 0.8 representing the non-linearity factor of the attenuation curve.

6. The virtual power plant cooperative control method based on multi-agent hierarchical reinforcement learning as described in claim 5, characterized in that: A virtual power plant (VPP) includes a resource access and aggregation module, a core control module, a market trading module, an operation monitoring module, and an interaction module. The resource access and aggregation module includes the access of distributed power sources and the management of flexible loads. Distributed power sources are used to integrate photovoltaic, wind power, energy storage or other distributed energy sources and support the IEEE 1547 / IEC 61850 communication protocol. The management of flexible loads includes accessing adjustable loads, including air conditioning clusters, charging piles or industrial production lines, and adjusting the load curve through demand response protocols.

7. The virtual power plant cooperative control method based on multi-agent hierarchical reinforcement learning as described in claim 6, characterized in that: The core control module includes a collaborative optimization engine and a prediction subsystem. The collaborative optimization engine optimizes the scheduling strategy in real time based on electricity price signals, grid constraints and resource characteristics. It uses linear programming algorithm and reinforcement learning algorithm to control the response delay within 300ms. The forecasting subsystem uses an LSTM model to forecast renewable energy output with an accuracy greater than 93%. It also forecasts electricity prices and load demand based on real-time electricity market prices.

8. The virtual power plant cooperative control method based on multi-agent hierarchical reinforcement learning as described in claim 6, characterized in that: The market trading module includes spot market bidding and ancillary services market. Spot market bidding aggregates resources to participate in electricity bidding transactions to maximize arbitrage profits. The ancillary services market provides frequency regulation AGC and reserve capacity assistance, with a frequency regulation response speed of less than 3 seconds.

9. The virtual power plant cooperative control method based on multi-agent hierarchical reinforcement learning as described in claim 6, characterized in that: The operation monitoring module includes a digital twin platform and an anomaly diagnosis system. The digital twin platform constructs a power grid indoor mapping to simulate extreme scenarios such as typhoons or faults, and outputs emergency plans and safety boundaries. The anomaly diagnosis system uses time-series data analysis and graph neural networks (GNNs) to locate equipment faults using knowledge graphs.

10. The virtual power plant cooperative control method based on multi-agent hierarchical reinforcement learning as described in claim 6, characterized in that: The interaction module includes a terminal control interface, which sends charging / discharging commands or start / stop commands to the resource owner.

Images