A virtual power plant cooperative operation optimization method and system based on double-layer game

Abstract

The application discloses a kind of virtual power plant collaborative operation optimization method and system based on double-layer game, method includes: the double-layer game framework of virtual power plant including dispatch control center, energy aggregator and load aggregator is established, wherein, dispatch control center is leader, energy aggregator is follower, and load aggregator is cooperative game participant;Establish load aggregator optimization model;Establish energy aggregator optimization model;Establish dispatch control center optimization model;Adjust the parameters of load aggregator optimization model, energy aggregator optimization model and dispatch control center optimization model, so that each model meets its corresponding constraint condition under the premise, reaches optimization target;Optimization control is carried out to actual power plant using the parameters of each model obtained after reaching optimization target;The application has the advantages that: the optimization control of internal multi-agent benefit coordination balance of virtual power plant is realized.

Description

Technical Field

[0001] This invention relates to the field of virtual power plant optimization operation and energy management, specifically to a method and system for optimizing the collaborative operation of virtual power plants based on two-level game theory. Background Technology

[0002] Virtual Power Plants (VPPs), as an innovative operating model integrating distributed energy resources, have received widespread attention and rapid development in recent years. VPPs aggregate geographically dispersed distributed generator sets, energy storage systems, controllable loads, and other resources through advanced information and communication technologies and intelligent control systems, enabling them to participate in electricity market transactions and grid dispatch as a whole. Their core advantages include the ability to aggregate small-capacity, dispersed resources into large-scale virtual units to improve market participation capabilities, coordinating the operation of various resources through optimization algorithms to maximize overall benefits, quickly responding to grid dispatch instructions to provide ancillary services such as peak shaving and frequency regulation, and participating in the electricity market as a single entity to simplify transaction processes. For example, Chinese Patent Publication No. CN116706960A discloses a multi-entity game control strategy for integrating wind, solar, and energy storage in a virtual power plant.

[0003] Despite significant progress in virtual power plant technology, key issues remain in existing research and applications: existing research often takes a single perspective, such as only considering the profit maximization of VPP operators or only focusing on grid-side dispatch optimization, neglecting the coordination of interests among various stakeholders within the VPP (power generators, load aggregators, users, etc.). This single-objective optimization often leads to the loss of interests of some stakeholders, affecting the sustainable operation of the VPP. Summary of the Invention

[0004] The technical problem to be solved by this invention is how to provide an optimization method that can achieve the coordination and balance of interests among multiple stakeholders within a virtual power plant.

[0005] This invention solves the above-mentioned technical problems through the following technical means: a virtual power plant collaborative operation optimization method based on two-layer game theory, comprising:

[0006] S1. Establish a two-layer game framework for a virtual power plant that includes a dispatch control center, energy aggregators, and load aggregators. The dispatch control center is the leader, the energy aggregator is the follower, and the load aggregator is the cooperative game participant. S2. Establish a load aggregator optimization model; S3. Establish an energy aggregator optimization model; S4. Establish an optimization model for the scheduling and control center; S5. Adjust the parameters of the load aggregator optimization model, the energy aggregator optimization model, and the dispatch control center optimization model so that each model achieves the optimization objective under the premise of satisfying its corresponding constraints; use the parameters of each model obtained after achieving the optimization objective to optimize the control of the actual power plant. The adjusted parameters of the energy aggregator optimization model are the active power output of thermal power units, wind power units, and photovoltaic systems.

[0007] This invention establishes an optimization model for load aggregators, an optimization model for energy aggregators, and an optimization model for dispatch control centers. It coordinates and optimizes the parameters of the three models so that all three models achieve their optimization objectives. The optimized parameters of each model are then used to guide the optimization and adjustment of actual power plants, achieving a balance of interests among multiple stakeholders.

[0008] Further, S1 includes: Define the elements of a game ,in For the participants to gather, These represent three decision-making entities: the dispatch control center, the energy aggregator, and the load aggregator, respectively. The strategy space contains the set of decision variables for each participant; To optimize the objective function set, each participant's strategy is mapped to its payoff value; the strategy space of each participant is defined as follows:

[0009]

[0010]

[0011] in, This represents the price decision vector of the dispatch control center, which includes electricity price settings for 24 time periods. Time periods The transaction price set by the dispatch control center for thermal power units, wind power units, photovoltaic systems and load aggregators; This represents the output decision vector of the energy aggregator; They represent time periods respectively. Active power output of thermal power units, wind power units, and photovoltaic systems This represents the response decision vector of the load aggregator; Indicates time period The demand response price that load aggregators offer to users.

[0012] Furthermore, S2 includes: The constraints and objective function of the load aggregator are set, and the two together constitute the load aggregator optimization model. The constraints of the load aggregator include... ,in, The minimum satisfaction threshold, Representative time period User overall satisfaction index and , and These are respectively the load reduction satisfaction index and the economic benefit satisfaction index. and These are the weighting coefficients for load reduction satisfaction and economic benefit satisfaction, respectively, to meet... ; The objective function of the load aggregator is: , This represents the comprehensive objective function value of the load aggregator; This is the economic benefit weighting coefficient. The user satisfaction weighting coefficient satisfies , Let be the economic return function. The user satisfaction function is shown in the following formula.

[0013] in, For time period The electricity price that load aggregators sell to users; For time period The actual electricity sales volume; For time period The electricity purchase price that load aggregators obtain from the dispatch control center; For time period Load following demand response implementation; The constraints on the load aggregator also include ,in, , These are the lower and upper limits of electricity sales prices, respectively. , These are the lower and upper limits of the load, respectively.

[0014] Further, S3 includes: Models for thermal power units, wind power units, and photovoltaic systems are set up to collectively constitute an energy aggregator optimization model. This energy aggregator optimization model serves as a grand alliance, defining the grand alliance... ,in Representing thermal power units, wind power units, and photovoltaic systems respectively, for any sub-alliance , characteristic function This indicates the maximum potential revenue that the sub-alliance can obtain:

[0015] in, For the alliance The characteristic function values; For the alliance The set of strategies; For alliance members The payoff function, The optimization objective function for the Major League Baseball is:

[0016] in, The strategy set for all power generation units. This represents the maximum total benefit that can be obtained when thermal power units, wind power units, and photovoltaic systems are optimized as a whole; the Shapley value method is used for benefit distribution.

[0017] in, For alliance members The Shapley value represents the alliance member. The distribution of profits due; For the alliance Number of members; The total number of members in the major leagues. To exclude alliance members The characteristic function values ​​of the alliance; The factorial symbol, For combined weights.

[0018] Furthermore, the thermal power unit model includes: The power generation cost function of a thermal power unit is:

[0019] in, For time period The power generation cost of thermal power units; For time period The generating capacity of thermal power units This is a secondary cost coefficient; This is the primary cost coefficient; This is the fixed cost coefficient; This refers to the start-up and shutdown cost coefficient. For time period The unit start-up and shutdown status variables, Indicates that the unit is in operation. This indicates that the unit is shut down; the relationship between carbon emissions and power generation of a thermal power unit is as follows:

[0020] in, For time period Carbon emissions; This represents the number of types of greenhouse gases emitted during the power generation process; For the first The fixed emission coefficient of greenhouse gases, For the first Variable emission factors for various greenhouse gases; thermal power units must meet the following technical constraints:

[0021] in, To achieve the minimum technical output of thermal power units; To contribute the maximum technical strength to thermal power units; To limit the rate of ascent; To limit the rate of ascent; the revenue function of the thermal power unit is:

[0022] in, The daily profit of thermal power units; For time period On-grid electricity price for thermal power units; The cost of fuel per unit of electricity generation; The cost of carbon emissions is calculated based on carbon trading prices and actual emissions.

[0023] Furthermore, the wind turbine model includes: The relationship between wind power and wind speed is described by a power curve:

[0024] in, For time period Wind turbines at wind speed Power generation under the given conditions; For time period The actual wind speed; To cut in wind speed; Rated wind speed; Cut off the wind speed; This refers to the rated power of the wind turbine generator set; The revenue function of a wind turbine is:

[0025] in, For the expected profit of wind turbine units, Indicates a scene, For all scene sets; For the scene The probability of; For time period The feed-in tariff for wind power, For time period The power generation capacity of wind turbine units; To calculate the decommissioning benefits over the scheduling cycle; For operation and maintenance costs; The cost is penalized for power deviation.

[0026] Furthermore, the photovoltaic system model includes: Photovoltaic power generation depends on sunlight intensity and ambient temperature:

[0027] in, For time period The power generation capacity of the photovoltaic system; The conversion efficiency of photovoltaic modules; This refers to the total area of ​​the photovoltaic panels; For time period Light intensity; For temperature coefficient, For time period Ambient temperature; For reference temperature; The payoff function for a photovoltaic system is:

[0028] in, Expected profits of photovoltaic units Indicates a scene, For all scene sets; For the scene The probability of; For time period The feed-in tariff for photovoltaic systems; To calculate the decommissioning benefits over the scheduling cycle; For operation and maintenance costs; The cost is penalized for power deviation.

[0029] Furthermore, S4 includes: The constraints and objective function of the scheduling control center are set, and the two together constitute the optimization model of the scheduling control center. The objective function of the scheduling control center is: ,in, For the daily revenue of the dispatch and control center; For time period Electricity sales revenue and , For time period The electricity price sold to load aggregators, For time period Load demand; For time period The cost of purchasing electricity and , , , Time periods The purchase price of electricity from thermal power, wind power, and solar power. , , For the output of each power generation unit; For time period The interaction cost with the external power grid and , For time period Electricity trading prices with external power grids; For time period Interaction power with the external power grid; This is the network loss coefficient; For time period Total output of energy aggregators; The constraints of the dispatch control center include price constraints, power balance constraints, and grid interaction capacity constraints; among which the price constraint is:

[0030] In the formula, For time period The lower limit of the price for purchasing electricity from external power grids; For time period Price ceiling for selling electricity to external power grids; Power balance constraint is

[0031] Grid interaction capacity constraints are

[0032] In the formula, This is the maximum transmission capacity of the connection line to the external power grid.

[0033] Further, S5 includes: The dispatch control center, energy aggregator, and load aggregator are modeled as three interacting agents. An Actor-Critic network architecture is set up for each agent. The Actor network is used to generate corresponding policy outputs based on the current environmental state; the policy is the agent's parameter adjustment scheme. The Critic network is used to evaluate the Q-value corresponding to the state-policy pair, i.e., the optimization objective function value. In each training iteration, the frequency is... Update the parameters in the scheduling control center, in terms of frequency. Update the parameters of the energy aggregator and load aggregator. The process continues to optimize and update until each agent achieves its optimization goal while satisfying its corresponding constraints.

[0034] This invention also provides a system for implementing the above-described virtual power plant collaborative operation optimization method based on two-level game theory, comprising: The game framework building module is used to establish a two-layer game framework for a virtual power plant, which includes a dispatch control center, energy aggregators, and load aggregators. In this framework, the dispatch control center is the leader, the energy aggregator is the follower, and the load aggregator is the cooperative game participant. The first model building module is used to build the load aggregator optimization model; The second model building module is used to build an energy aggregator optimization model. The third model building module is used to build the optimization model of the scheduling and control center. The optimization solution module is used to adjust the parameters of the load aggregator optimization model, the energy aggregator optimization model, and the dispatch control center optimization model so that each model achieves the optimization objective while meeting its corresponding constraints. The parameters of each model obtained after achieving the optimization objective are used to optimize the control of the actual power plant. The adjusted parameters of the energy aggregator optimization model are the active power output of thermal power units, wind power units, and photovoltaic systems.

[0035] The advantages of this invention are: (1) This invention establishes a load aggregator optimization model, an energy aggregator optimization model, and a dispatch control center optimization model, and coordinates and optimizes the parameters of the three models so that all three models achieve their optimization objectives. Thus, the optimized parameters of each model are used to guide the optimization and adjustment of actual power plants, thereby achieving a balance of interests among multiple stakeholders.

[0036] (2) The load aggregator optimization model of the present invention constructs a two-dimensional user satisfaction evaluation system that integrates load reduction satisfaction and economic benefit satisfaction. It also introduces a price elasticity matrix containing self-elasticity coefficient and cross-elasticity coefficient to fully characterize the coupled impact of electricity price changes on electricity consumption at different times. At the same time, it sets a minimum satisfaction threshold constraint to ensure that users' basic electricity needs are not excessively sacrificed in the process of pursuing maximum economic benefits, and to achieve a sustainable balance between economic benefits and user experience. The energy aggregator optimization model introduces a carbon emission trading mechanism in thermal power units, and directly incorporates carbon trading costs into the revenue function so that power generation decisions can respond to dual price signals from the electricity market and the carbon trading market. Simultaneously, a cooperative game model is established for three types of power generation units: thermal power, wind power, and photovoltaic power. The Shapley value method is used to fairly distribute revenue based on the marginal contributions of each participant, fully leveraging the spatiotemporal complementarity of heterogeneous power generation resources. The dispatch control center optimization model, acting as the leader in the Stackelberg game, explicitly includes the optimal response functions of lower-level energy aggregators and load aggregators in its objective function. This allows for anticipating the response behavior of lower-level participants when formulating electricity pricing strategies and making forward-looking decisions accordingly. Furthermore, differentiated transaction prices are set for different types of power generation resources and load aggregators, enabling targeted guidance of various resources through refined price signals. These three models are coupled and synergistically optimized through a two-layer game framework, jointly achieving multi-objective coordination of the virtual power plant's economic benefits, environmental benefits, and user satisfaction.

[0037] (3) This invention effectively coordinates the conflicting objectives of multiple stakeholders, including the dispatch center, the power generation side (energy aggregators), and the power consumption side (load aggregators and end users), through the Stackelberg-cooperative game hybrid framework. This avoids the imbalance of interests caused by single-objective optimization and ensures the long-term stability of the virtual power plant operation. Simulation experiments show that this method can effectively reduce system operating costs, significantly improve the utilization rate of renewable energy, and significantly reduce carbon emissions, achieving a balance of economic, environmental, and social benefits.

[0038] (4) This invention takes into account both economic benefits and user experience. It introduces a quantitative user satisfaction model on the demand side and integrates it into the optimization process as a hard constraint. This ensures that while implementing demand response and reducing system costs, the user's electricity experience is guaranteed or even improved, thereby enhancing the acceptability and sustainability of the strategy.

[0039] (5) The optimization solution algorithm proposed in this invention can effectively handle complex game problems with high dimensions and nonlinearity. Compared with traditional algorithms, it has a faster convergence speed and better stability, which meets the engineering application requirements of real-time or near-real-time optimization scheduling of virtual power plants. Attached Figure Description

[0040] Figure 1This is an architecture diagram of a virtual power plant collaborative operation optimization method based on two-layer game theory disclosed in an embodiment of the present invention; Figure 2 This is a flowchart illustrating the optimization solution of a virtual power plant collaborative operation optimization method based on two-layer game theory, as disclosed in an embodiment of the present invention. Figure 3 This is a performance comparison chart of four optimization schemes in a virtual power plant collaborative operation optimization method based on two-layer game theory disclosed in an embodiment of the present invention. Figure 4 The 24-hour dynamic electricity price change curve is shown in the virtual power plant collaborative operation optimization method based on two-layer game theory disclosed in this embodiment of the invention. Figure 5 The curve showing the 24-hour user satisfaction variation in a virtual power plant collaborative operation optimization method based on a two-layer game theory disclosed in an embodiment of the present invention. Figure 6 This is a comparison chart of the convergence performance of algorithms in a virtual power plant collaborative operation optimization method based on two-layer game theory disclosed in an embodiment of the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] Example 1 While some mainstream virtual power plant (VPP) optimization schemes incorporate game theory, most employ only simple non-cooperative games or static games with complete information, failing to accurately characterize the hierarchical decision-making structure and dynamic interaction processes within VPPs. In particular, they lack a comprehensive framework combining Stackelberg games with cooperative games. Existing optimization models often treat users as passive electricity consumers, neglecting the impact of user satisfaction and comfort on the sustainability of demand response strategies, and lacking effective user incentive mechanisms and satisfaction guarantees. VPP optimization problems are characterized by high dimensionality, nonlinearity, and multiple constraints, making them difficult to solve using traditional mathematical programming methods. Existing heuristic algorithms have slow convergence speeds and are prone to getting trapped in local optima, making them unsuitable for real-time scheduling. In the context of "dual carbon" (carbon and environmental), carbon emission costs are increasingly important, but existing research rarely incorporates carbon trading mechanisms into VPP optimization models, lacking a synergistic optimization of economic incentives and environmental benefits.

[0043] like Figure 1 and Figure 2As shown, Embodiment 1 of this invention provides a virtual power plant collaborative operation optimization method based on a two-layer game theory approach. By establishing a Stackelberg-cooperative game hybrid framework and designing the H-MADDPG reinforcement learning algorithm, it achieves efficient collaborative operation of the virtual power plant, aiming to solve the following key technical problems: First, the multi-level, multi-agent coordination optimization problem, i.e., how to establish an optimization framework that can simultaneously consider the interests of multiple stakeholders such as the dispatch control center, energy aggregators, and load aggregators, and achieve a coordinated balance of interests among all parties. This requires solving complex problems such as conflicts in the objective functions of different stakeholders, information asymmetry, and differences in decision-making timing. Second, the game equilibrium solution problem, i.e., how to efficiently solve the game equilibrium problem involving Stackelberg... The complex equilibrium problems of Ackerberg games and cooperative games, especially in large-scale, high-dimensional real-world systems, face the challenges of dimensionality curse and computational complexity explosion from traditional solution methods. Thirdly, there is the problem of balancing user satisfaction and economic benefits, i.e., how to maximize economic benefits while ensuring user satisfaction and comfort with electricity usage, and establishing a sustainable demand response mechanism. This requires accurate modeling of user behavior characteristics and designing reasonable incentive mechanisms. Fourthly, there is the problem of synergistic development between renewable energy consumption and carbon emission reduction, i.e., how to improve the utilization rate of renewable energy through optimized scheduling while reducing system carbon emissions. This requires comprehensive consideration of technical constraints, economic costs, and environmental benefits to achieve multi-objective synergistic optimization. To solve the above technical problems, this invention provides a comprehensive optimization method that can achieve a balance of interests among multiple stakeholders within a virtual power plant, improve system operating efficiency, promote renewable energy consumption, and reduce carbon emissions. The method includes the following steps: S1: Constructing a two-layer game framework for a virtual power plant The purpose of this step is to establish an overall framework for the coordinated optimization of the virtual power plant, laying the foundation for the subsequent establishment of various entity models. A two-tiered virtual power plant system is established, comprising a Dispatch Control Center (DCC), Energy Aggregators (EAs), and Load Aggregators (LAs). The Dispatch Control Center, acting as the leader in the Stackelberg game, is responsible for setting dynamic electricity price signals, coordinating energy trading, managing power interaction with the external grid, and monitoring system operation to ensure safety and stability. Energy Aggregators, as followers and cooperative game participants, manage thermal power units (providing base load and peak-shaving services), wind power units (contributing clean energy but with uncertain output), and photovoltaic systems (generating electricity during the day, affected by sunlight and temperature). Load Aggregators, as followers, are responsible for collecting and analyzing user electricity consumption information, implementing demand response strategies, and ensuring user satisfaction. Game elements are defined. ,in For the participants to gather, These represent three decision-making entities: the dispatch control center, the energy aggregator, and the load aggregator, respectively. The strategy space contains the set of decision variables for each participant; Let be the set of payoff functions, mapping each participant's strategy to its payoff value. The strategy space for each participant is defined as follows: (1) (2) (3) Formulas (1)-(3) above define the decision variables of each participating entity in the virtual power plant, wherein, in formula (1) This represents the price decision vector of the dispatch control center, which includes electricity price settings for 24 time periods; Time periods ( (This refers to the transaction price set by the dispatch and control center for thermal power units, wind power units, photovoltaic systems, and load aggregators during the 24 hours of a day, expressed in yuan / kWh.) This represents the feasible strategy space of the scheduling control center, constrained by upper and lower price limits. In formula (2), This represents the output decision vector of the energy aggregator; They represent time periods respectively. Active power output of thermal power units, wind power units, and photovoltaic systems, in MW; This represents the feasible strategy space for energy aggregators, which is limited by the technical constraints of each unit. In formula (3), This represents the response decision vector of the load aggregator; Indicates time period Demand response electricity price offered by load aggregators to users, in yuan / kWh; This represents the feasible strategy space for load aggregators, constrained by the upper and lower limits of electricity prices.

[0044] Payoff function Map each subject's strategy to a payoff value, and define the leader's payoff function: (4) Map the strategy combination of the scheduling control center and lower-level followers to the leader's payoff; define the follower payoff function: (5) (6) The strategy combinations are mapped to the payoffs of energy aggregators and load aggregators, respectively. Through this game theory framework, each participant optimizes their decisions within their respective strategy spaces, and multi-agent coordination is achieved through the interrelation of payoff functions. This game theory framework provides a unified mathematical description and constraint structure for establishing the specific optimization models for each agent in subsequent steps S2 to S4.

[0045] S2: Establish a refined model for the load aggregator. Building upon the game theory framework established in step S1, this step establishes a refined optimization model for the load aggregator to coordinate demand response mechanisms with user satisfaction. The load aggregator guides users to adjust their electricity consumption behavior through the demand response mechanism, the core of which is based on price elasticity theory. Price elasticity matrix : (7) The price elasticity matrix This describes the user's response characteristics to changes in electricity prices and is a key parameter in the transformation from the game theory framework of step S1 to a specific demand response model. The matrix consists of diagonal and off-diagonal elements. The elasticity coefficient reflects the impact of electricity price changes on electricity consumption during the same period. A negative elasticity coefficient indicates that an increase in electricity price will lead to a decrease in electricity consumption. In this embodiment, the elasticity coefficient is taken as -0.15 during peak hours, -0.20 during normal hours, and -0.25 during valley hours. (Off-diagonal elements) The cross-elasticity coefficient represents the effect of electricity price changes in time period j on the price changes in time period j. The load impact reflects the load transfer effect between different time periods. A positive cross-price elasticity coefficient indicates that users will shift load from high-price periods to low-price periods. In this embodiment, the peak-valley cross-price elasticity coefficient is taken as 0.10, and the cross-price elasticity coefficient between adjacent time periods is taken as 0.05. The price elasticity coefficient is defined as the ratio of the rate of change in electricity consumption to the rate of change in electricity price. (8) in, For time period Changes in electricity consumption (unit: MW) For time period The benchmark electricity consumption (unit: MW). For time period Changes in electricity prices (unit: yuan / kWh) For time period The benchmark electricity price (unit: yuan / kWh). Based on the price elasticity matrix, the load vector after implementing demand response is: (9) This formula establishes a quantitative relationship between electricity price changes and load response, and is the core model connecting the load aggregator strategy variables in step S1 with actual load adjustments. Among them, , Indicates A diagonal matrix with diagonal elements. Electricity price change rate vector. The Each component is defined as: (10) in, The price is expressed as the electricity price after demand response is implemented in time period t (unit: yuan / kWh). The benchmark electricity price for time period t is expressed as (unit: yuan / kWh). The electricity price change rate is dimensionless. Based on the demand response mechanism, a user satisfaction model is further established to ensure the sustainability of the demand response strategy. Overall user satisfaction. Defined as: (11) in, Representative time period The overall user satisfaction index, the value range of which is: The higher the value, the higher the user satisfaction. and These are respectively the load reduction satisfaction index and the economic benefit satisfaction index. and These are the weighting coefficients for load reduction satisfaction and economic benefit satisfaction, respectively, to meet... To ensure users' basic electricity needs are met, a satisfaction constraint is set: (12) in, In this embodiment, the minimum satisfaction threshold is set to... Based on the above demand response model and satisfaction model, the optimization objective function of the load aggregator is: (13) The objective function comprehensively considers economic benefits and user satisfaction, thus achieving the revenue function of the load aggregator in step S1. The specific details. Among them, This represents the comprehensive objective function value of the load aggregator; This is the economic benefit weighting coefficient. The user satisfaction weighting coefficient satisfies In this embodiment, we take , ; Let be the economic return function. For user satisfaction function. For economic benefit function. and user satisfaction function The specific form is as follows: (14) in, For time period Electricity prices sold by load aggregators to users (unit: yuan / kWh); For time period Actual electricity sales volume (unit: MWh); For time period The electricity purchase price (unit: yuan / kWh) from the dispatch control center by the load aggregator is determined by the dispatch control center's strategy in step S1; For time period The load after implementing demand response is calculated using formula (9); For time period The overall user satisfaction is calculated using formula (11). The load aggregator must meet the following constraints: (15) in, , These represent the lower and upper limits of electricity sales prices (unit: yuan / kWh). , These represent the lower and upper limits of the load (unit: MW). The load aggregator model established in this step realizes a complete mapping from the load aggregator strategy space in the game framework of step S1 to specific optimization objectives and constraints, providing an optimization sub-problem on the load side for solving the game equilibrium in the subsequent step S5.

[0046] S3: Establish an energy aggregator optimization model Based on the game theory framework established in step S1 and the load model established in step S2, this step establishes an optimization model for the energy aggregator to achieve coordinated optimization of power generation resources on the supply side. The energy aggregator coordinates and manages three types of power generation resources: thermal power, wind power, and photovoltaic power, and the output of each resource must match the load demand in step S2.

[0047] (1) Thermal power unit model Thermal power units, as controllable and stable power sources, undertake the dual tasks of base load and peak shaving in virtual power plants. The power generation cost function of thermal power units is: (16) in, For time period Electricity generation cost of thermal power units (unit: yuan); For time period The power generation capacity of thermal power units (unit: MW) belongs to the energy aggregator strategy variable defined in step S1; This is a secondary cost coefficient (unit: yuan / MW²), reflecting the characteristic that power generation efficiency decreases as output increases. In this embodiment, it is taken as... Yuan / MW²; This is the primary cost coefficient (unit: yuan / MW), corresponding to the marginal cost of fuel. In this embodiment, it is taken as... Yuan / MW; This is the fixed cost factor (unit: yuan), which includes fixed expenses such as equipment depreciation and labor. In this embodiment, it is taken as... Yuan; The start-stop cost coefficient (unit: yuan) is taken in this embodiment. Yuan; For time period The unit start-up and shutdown status variables, Indicates that the unit is in operation. This indicates that the unit is shut down. Under the dual-carbon context, thermal power units need to consider carbon emission costs. The relationship between carbon emissions and power generation is as follows: (17) in, For time period Carbon emissions (unit: kg); This represents the number of types of greenhouse gases emitted during the power generation process; For the first The fixed emission factor of greenhouse gases (unit: kg) is taken in this embodiment. For the first The variable emission factor of the greenhouse gas (unit: kg / MW) is taken in this embodiment. Thermal power units must meet the following technical constraints: (18) in, The minimum technical output of a thermal power unit (unit: MW) is taken as 200MW in this embodiment; The maximum technical output of the thermal power unit (unit: MW) is taken as 600MW in this embodiment; To limit the upward climbing speed (unit: MW / h), 100MW / h is used in this embodiment; To limit the downward ramp rate (unit: MW / h), a rate of 100 MW / h is used in this embodiment. The revenue function of the thermal power unit is: (19) in, Daily profit of thermal power units (unit: yuan); For time period The on-grid electricity price of thermal power units (unit: yuan / kWh) is determined by the strategy of the dispatch control center in step S1; The unit fuel cost for power generation (unit: yuan / kWh) is calculated by the power generation cost function in formula (16), that is, by dividing the power generation cost calculated by formula (16) by the power generation power of the corresponding period, thereby obtaining the fuel cost per unit of power generation. In this embodiment, it is taken as 0.30 yuan / kWh; The carbon emission cost (unit: yuan) is calculated by formula (17), that is, the carbon emission cost is equal to the sum of the carbon trading price and the carbon emission amount for each period obtained by formula (17).

[0048] (2) Wind turbine model The relationship between wind power and wind speed is described by a power curve: (20) in, For time period Wind turbines at wind speed The power generation under the given conditions (unit: MW) belongs to the energy aggregator strategy variable defined in step S1; For time period Actual wind speed (unit: m / s); The cut-off wind speed is the minimum wind speed at which the wind turbine starts generating electricity (unit: m / s), which is taken as 3 m / s in this embodiment; Rated wind speed, which is the wind speed when the wind turbine reaches its rated power (unit: m / s), is taken as 12 m / s in this embodiment; The cut-off wind speed is the highest wind speed (unit: m / s) that protects the equipment and prevents the wind turbine from generating electricity. In this embodiment, it is taken as 25 m / s. The rated power of the wind turbine is 400MW in this embodiment.

[0049] The revenue function of a wind turbine is: (twenty one) in, Expected profit of wind turbine units (unit: yuan). Indicates a scene, For all scene sets; For the scene The probability of; For time period The on-grid price of wind power (unit: yuan / kWh) is determined by the dispatch and control center; Decommissioning revenue converted to the scheduling cycle (unit: yuan); Operating and maintenance costs (unit: yuan); This is the power deviation penalty cost (unit: yuan), which is incurred when the actual output deviates significantly from the planned output.

[0050] (3) Photovoltaic system model Photovoltaic power generation capacity mainly depends on sunlight intensity and ambient temperature: (twenty two) in, For time period The power generation of the photovoltaic system (unit: MW) is a variable of the energy aggregator strategy defined in step S1; The conversion efficiency (dimensionless) of the photovoltaic module is taken as 18% in this embodiment; The total area of ​​the photovoltaic panels (unit: m²) is taken in this embodiment. ; For time period Light intensity (unit: W / m² or kW / m²). The temperature coefficient (unit: 1 / °C) reflects the characteristic that efficiency decreases as temperature increases. In this embodiment, it is taken as... For time period Ambient temperature (unit: °C); The reference temperature (unit: °C) is usually taken as 25°C.

[0051] The payoff function for a photovoltaic system is: (twenty three) in, Expected profit of photovoltaic units (unit: yuan). Indicates a scene, For all scene sets; For the scene The probability of; For time period The feed-in tariff for photovoltaic systems (unit: yuan / kWh) is determined by the dispatch and control center; Decommissioning revenue converted to the scheduling cycle (unit: yuan); Operating and maintenance costs (unit: yuan); This is the power deviation penalty cost (unit: yuan), which is incurred when the actual output deviates significantly from the planned output.

[0052] (4) Cooperative Game Model of Energy Aggregators To fully leverage the complementary characteristics of different power generation units, a cooperative game theory model is established to achieve synergistic optimization of power generation resources. This defines a large alliance. ,in These represent thermal power units, wind power units, and photovoltaic systems, respectively. For any sub-alliance... , characteristic function This indicates the maximum potential revenue that the sub-alliance can obtain: (twenty four) in, For the alliance The characteristic function value (unit: yuan); For the alliance The set of strategies; For alliance members The payoff function, The optimization objective function for the Major League Baseball is: (25) in, This is the set of strategies for all power generation units. This represents the maximum total benefit that can be obtained when thermal power units, wind power units, and photovoltaic systems are optimized as a whole, i.e., the characteristic function value of the grand alliance. A set of synergistic optimization strategies among the three By complementing each other (such as using thermal power to shave off peak power when wind and solar power output is insufficient), the overall total benefit is maximized, typically exceeding the sum of the benefits from individual optimization. To ensure the stability and fairness of the cooperative game, the Shapley value method is used for benefit distribution. (26) in, For alliance members The Shapley value (in dollars) represents the value of a member of the alliance. The distribution of profits due; For the alliance Number of members; The total number of members in the major league, here ; The characteristic function value of the alliance S; For those who do not include participants The characteristic function values ​​of the alliance; The factorial symbol, For the combined weights, participants were taken into account. The probability of joining various possible alliance orders. The Shapley value method reflects the marginal contribution of each participant to the alliance, satisfying fairness principles such as efficiency, symmetry, virtuality, and additivity.

[0053] It should be noted that the Shapley value does not directly participate in the construction of the large alliance's optimization objective function. The large alliance's optimization objective function (Formula 25) aims to maximize the total collaborative revenue of the three types of power generation units, achieving overall optimality through a unified set of optimization strategies. The Shapley value (Formula 26) is applied to the revenue distribution stage after optimization. By calculating the marginal contribution of each power generation unit in all possible sub-alliance combinations, the total revenue of the large alliance is fairly distributed among the thermal power units, wind power units, and photovoltaic systems, ensuring that the revenue received by each participant matches their actual contribution to the alliance. In the H-MADDPG algorithm in step S5 below, the above Shapley value allocation result is embedded in the reward function of the energy aggregator agent, so that each power generation unit uses its Shapley value allocation share as an individual reward signal during the strategy learning process. This ensures the fairness and stability of revenue distribution in the cooperative game while pursuing the maximization of the overall revenue of the large alliance. In general, the optimization objective of the large alliance is that thermal power, wind power, and photovoltaics first coordinate to maximize the total profit, and then the profit distribution is determined according to the Shapley value. The sum of the profits of the three equals the total profit.

[0054] The energy aggregator model established in this step achieves a complete mapping from the energy aggregator strategy space in the game framework of step S1 to the optimization of specific power generation resources, and realizes collaborative optimization among power generation resources through a cooperative game mechanism. The output of this model (the output of each power generation unit) needs to achieve a supply-demand balance with the load demand of the load aggregator model in step S2. This balance will be coordinated as a core constraint in the dispatch control center model in step S4.

[0055] S4: Establish an optimization model for the scheduling and control center. Building upon steps S1 to S3, this step establishes an optimization model for the dispatch control center as the leader in the Stackelberg game. The dispatch control center coordinates the load aggregators in step S2 and the energy aggregators in step S3 through electricity price signals to achieve supply and demand balance and maximize overall profit. The dispatch control center's revenue primarily comes from the price difference in electricity trading, i.e., profiting by buying low and selling high. Simultaneously, when there is an internal supply and demand imbalance, the interaction costs with the external power grid need to be considered. The optimization objective function of the dispatch control center is: (27) in, Daily revenue of the dispatch and control center (unit: yuan); For time period Electricity sales revenue (unit: yuan) comes from electricity sales to load aggregators; For time period The cost of electricity purchase (unit: yuan) comes from purchasing electricity from energy aggregators; For time period Interaction costs with the external power grid (unit: yuan). The specific calculations for each revenue and cost item are as follows: (1) Electricity sales revenue: (28) in, For time period Electricity price sold to load aggregators (unit: yuan / kWh) For time period The load demand (unit: MW) is calculated by the model in step S2.

[0056] (2) Electricity purchase cost: (29) in, , , Time periods Electricity purchase price from thermal power, wind power, and photovoltaic power (unit: yuan / kWh) , , The output of each power generation unit (unit: MW) is calculated by the model in step S3.

[0057] (3) Grid interaction cost: (30) in, For time period The transaction price with the external power grid (unit: yuan / kWh) is 0.65 yuan / kWh for the purchase price and 0.45 yuan / kWh for the sale price in this embodiment; For time period Interaction power with the external power grid (unit: MW), positive value indicates purchasing electricity from the grid, negative value indicates selling electricity to the grid; The network loss coefficient (dimensionless) is taken as 0.05 in this embodiment; For time period Total output of energy aggregators (unit: MW). The dispatch control center must meet the following constraints to ensure the safe and stable operation of the system and a fair and orderly market: (1) Price constraints: (31) in, For time period Lower limit of the price for purchasing electricity from external power grids (unit: yuan / kWh); For time period The price ceiling for selling electricity to the external power grid (unit: yuan / kWh); these constraints ensure that the transaction price within the virtual power plant does not deviate too much from the market price, maintaining the effectiveness of price signals and market equilibrium.

[0058] (2) Power balance constraint (32) in, For time period The load demand (unit: MW) is calculated by the load aggregator model in step S2; For time period Interaction power with external power grid (unit: MW); Time period The total output of the energy aggregator (unit: MW) is calculated from the energy aggregator model in step S3. The power balance constraint is a hard constraint and must be strictly satisfied. Any violation will lead to system frequency deviation, affecting power quality and even jeopardizing system safety. Therefore, this constraint has the highest priority and is the core link connecting the load side in step S2 and the supply side in step S3.

[0059] (3) Power grid interaction capacity constraints (33) in, The maximum transmission capacity (in MW) of the interconnection line with the external power grid is taken as 100MW in this embodiment. This constraint ensures that the interaction between the virtual power plant and the power grid will not exceed the transmission capacity of the interconnection line.

[0060] The dispatch control center model established in this step, acting as the leader in the Stackelberg game, guides the load aggregator in step S2 and the energy aggregator in step S3 to respond through electricity price signals, thus realizing the complete closed loop of the multi-agent game framework defined in step S1.

[0061] S5: Design an H-MADDPG solution algorithm Based on the two-layer game optimization model established in steps S1 to S4, this step designs the H-MADDPG (Hierarchical Multi-Agent Deep Deterministic Policy Gradient) reinforcement learning algorithm for solving the problem. Because this optimization problem is highly dynamic and structurally complex, traditional mathematical programming methods are difficult to solve effectively; therefore, it is incorporated into a multi-agent reinforcement learning framework for processing.

[0062] (1) Reinforcement learning environment modeling The dispatch control center, energy aggregator, and load aggregator are each modeled as three interacting agents, and the set of agents is defined as follows: Each intelligent agent continuously learns and optimizes its own strategies through interaction with the environment.

[0063] The system state space needs to comprehensively reflect the system's operating status and is defined as follows: (34) in, For time period The system state vector; For state space; For market status information, including grid electricity prices Load forecast Forecast values ​​of renewable energy output ; Resource status information, including thermal power output. Wind power output Photovoltaic power output Current load ; This is historical information, including electricity prices from the previous period. Electricity sales price Strategies of each agent , , This multi-dimensional state representation ensures that the agent has sufficient information to make decisions, incorporating the input and output variables of each model in steps S2 to S4 into a unified state space description.

[0064] (2) Definition of action space Each agent's action space reflects its decision-making authority in the virtual power plant, corresponding to the policy space defined in step S1: (35) in, For the actions of the dispatch and control center, including electricity purchase price and electricity sales price , This refers to the actions of energy aggregators, including the output decisions of each power generation unit. For the actions of load aggregators, including demand response load volume Electricity price for users , , , These are the action spaces for each agent.

[0065] (3) Reward function design The reward function is designed based on constrained optimization theory, and the penalty function method is used to incorporate the constraints of each model in steps S2 to S4 into the reward signal: (36) in, For intelligent agents In state Take action The reward value obtained later For intelligent agents The profit function corresponds to the profit function of each subject defined in step S1. . The penalty coefficient controls the severity of the penalty for constraint violation; To constrain the violation penalty, a positive penalty is generated when an action violates any of the constraints in steps S2 to S4. Specifically, the reward function for each agent corresponds to formula (27) in step S4, formula (25) in step S3, and formula (13) in step S2, respectively. Reward from the scheduling control center: (37) in, This includes penalties for violations of price constraints and power balance constraints. Penalty coefficient for the dispatch and control center. Energy aggregator reward: (38) in, This includes penalties for violating power limits and ramp rate constraints. Penalty coefficient for energy aggregators. Rewards for load aggregators: (39) in, This includes penalties for violations of satisfaction constraints and price constraints. This is the penalty coefficient for the load aggregator.

[0066] 4) Core Mechanism of H-MADDPG Algorithm To reflect the hierarchical structure of the Stackelberg game, the H-MADDPG algorithm employs the following core mechanism: a) Tiered update mechanism Different network update frequencies are set to reflect the hierarchical relationship between leaders and followers: the update frequency of the leader (dispatch control center). Update frequency greater than that of followers (energy aggregators and load aggregators) ,Right now This mechanism ensures that leaders can anticipate followers' responses, thereby formulating optimal pricing strategies. In this embodiment, we take... , .

[0067] b) Shapley value embedding Within the energy aggregator, the Shapley value allocation mechanism (Formula 26) from step S3 is embedded into the reward calculation to achieve fair distribution of revenue from the three types of power generation resources: thermal power, wind power, and photovoltaic power.

[0068] The specific mechanism by which Shapley value participates in reward calculation is as follows: During the training process of the H-MADDPG algorithm, the Energy Aggregator (EA) interacts with the environment as a whole intelligent agent, and its immediate reward is calculated by formula (38), that is... ,in The total cooperative benefit of the large alliance corresponds to formula (25). The Shapley value is embedded in the benefit redistribution stage after each training round (see algorithm iteration process): The algorithm, according to formula (26), traverses all possible sub-alliance combinations, calculates the marginal contribution of each power generation unit (thermal power, wind power, photovoltaic) when joining different sub-alliances, and obtains their respective Shapley values ​​φ_thermal power, φ_wind power, and φ_photovoltaic. Then, the total reward obtained by the EA agent is fairly distributed to the three types of power generation resources according to this ratio. The individual reward after distribution will serve as a feedback signal for the strategy optimization of each power generation unit in the next round of training, guiding each unit to adjust its output strategy in the direction of improving its own marginal contribution to the alliance, rather than simply pursuing the maximization of individual profit. Therefore, the Shapley value is embedded in the algorithm process as a "post-processing distribution mechanism" of the reward to ensure the fairness of the benefit distribution in cooperative game (satisfying efficiency, symmetry, dummy elements, and additivity), thereby ensuring the long-term cooperative willingness of each power generation resource and the stability of the alliance.

[0069] c) Lagrange multiplier method for handling constraints By introducing the Lagrange multiplier λ, the constrained optimization problem is transformed into an unconstrained problem: (40) in, For the first A constraint function, when the constraint is satisfied. When violated , Represents the Lagrange multipliers. Indicates the agent's state Take action The reward value obtained afterward.

[0070] (5) Actor-Critic Network Architecture The Actor-Critic architecture is constructed using deep neural networks: The Actor network (i.e., policy network) generates corresponding action outputs based on the current environmental state. Its structure includes: an input layer receiving state vectors with dimensions 32 to 48 (the specific dimension depends on the system configuration), followed by three fully connected hidden layers containing 256, 128, and 64 neurons respectively, each using the ReLU activation function; the output layer has the same dimension as the action space and uses the Tanh activation function to constrain the action output within the [-1, 1] interval. The Critic network (i.e., value network) evaluates the Q-value corresponding to a state-action pair. Its input layer integrates the state and action vectors, with the total input dimension being the sum of the state and action dimensions; the network body contains three fully connected hidden layers with 512, 256, and 128 neurons respectively, all using the ReLU activation function; the final output layer is a single-dimensional linear output used to estimate the expected cumulative reward value of the state-action pair.

[0071] (6) Algorithm Iteration Process This invention proposes a two-layer game-solving method based on the hierarchical multi-agent deep deterministic policy gradient (H-MADDPG) algorithm, specifically including the following steps: First, in the initialization phase, the Actor network parameters are initialized for each agent. and Critic network parameters And set the corresponding target network parameters. Simultaneously construct a capacity of 10 6 Experience replay pool The learning rates for the Actor network and the Critic network were set as follows: and Discount factor and the target network soft update coefficient Subsequently, during the environment interaction and data collection phase, each training round... to Perform the following operation: Reset the environment to its initial state. ; at each time step to Internally, the DCC agent operates according to its strategy. Based on the current state Select price action After observing the price signals released by DCC, the EA agent and the LA agent each formulate their own strategies. and Select the output action With response action The three parties jointly carried out the action. And observe the next state. and the corresponding instant rewards for each intelligent agent The experience of this quadruple Store in the experience replay pool In the middle. When the number of samples in the experience replay pool reaches the training requirement, the network enters the update phase: from Medium random sampling batch size Empirical samples; for EA and LA agents (each (Updated once per step), calculate the target. value ,in, This indicates the current system state, including the market state. Historical status and historical resource status , This represents the joint action performed by three agents in the current state, including the pricing action of the scheduling and control center. The efforts of energy aggregators The response actions of the load aggregator . This represents the immediate rewards received by each agent, including rewards from the scheduling and control center. Energy Aggregator Rewards and load aggregator rewards Instant rewards for each agent The reward of the scheduling control center is determined by its reward function and the penalty term for constraint violation. The energy aggregator reward corresponds to the electricity purchase and sale price difference revenue in formula (27) in step S4. The corresponding alliance collaborative benefit and load aggregator reward in formula (25) of step S3. The economic benefit and user satisfaction weighted target in formula (13) in step S2 correspond to the step S2. Represents the temporal difference objective value of each agent, including Used to update Critic network parameters. This represents the system state transitioned to at the next moment after the execution of a joint action, and its composition is similar to... same, This indicates that the objective policy network of each agent determines the state based on the next time step. The generated target action. Q represents the output value of the Critic network for each of the three agents, i.e. The long-term expected revenues of the dispatch control center, energy aggregator, and load aggregator under the current state-action pair are evaluated separately. The parameters of the Critic network represent the objectives of each of the three agents, i.e. The parameters of the main Critic network are updated via a soft update mechanism. Slowly track updates to ensure the stability of the training process. This represents the state of the Critic network for each agent in the next time step. and target action Q-value estimation under the following conditions, i.e. , , Used to calculate the target value . From target strategy Generate, and by minimizing the mean square error loss Update the Actor network; and for the DCC agent acting as the leader (each (Updated every time step), the calculation of the target Q-value needs to explicitly consider the optimal response behavior of the lower-level EA and LA agents under their latest policies, thus reflecting the hierarchical dependency of the Stackelberg game. Next, a soft update mechanism is used to synchronize the target network parameters: , After each training round, the Shapley value payout mechanism is implemented: within the EA agent, the marginal contribution of each power generation unit to the overall payout is calculated according to formula (26), and the total payout obtained by EA is fairly distributed according to the Shapley value ratio to ensure individual rationality and group efficiency in cooperative game. Finally, in the convergence judgment phase, the change between the current strategy and the previous iteration strategy is calculated. ;like Less than the preset convergence threshold (This embodiment takes) If the maximum number of iterations has been reached, the algorithm is considered converged and training is terminated; otherwise, it returns to the environment interaction stage to continue iterative optimization. Through the above steps, the H-MADDPG algorithm designed in this invention effectively solves the two-layer game model constructed in steps S1 to S4. Its hierarchical asynchronous update mechanism accurately depicts the decision-making sequence of the leader and followers in the Stackelberg game, while the embedded Shapley value allocation mechanism ensures the fairness and stability of the payoff distribution in the lower-level cooperative game.

[0072] It should be noted that the method proposed in this invention, when operating under a virtual power plant, adjusts the time period. ( (This refers to the trading price set by the dispatch control center for thermal power units, wind power units, photovoltaic systems, and load aggregators during the 24 hours of a day, and the adjustment period.) Active power output and adjustment periods of thermal power units, wind power units and photovoltaic systems The demand response price set by the load aggregator for users enables the three optimization models to achieve their optimization objectives. The aforementioned transaction price and demand response price are parameters continuously adjusted by the algorithm, not quantities that fluctuate with market economic conditions. After the entire algorithm optimization process of this invention is completed, the aforementioned parameters are fixed and directly applied to the actual power plant for optimized control. Therefore, the overall scheme is similar to a simulation optimization process under a virtual power plant, using the optimization results to guide actual operation. The following step S6 is only for verifying the effectiveness of the algorithm of this invention and is not within the scope of protection of this invention.

[0073] S6: Game Equilibrium Solving and Analysis Based on the H-MADDPG algorithm designed in step S5, this step solves and analyzes the game equilibrium to verify the convergence of the algorithm and the stability of the equilibrium solution.

[0074] (1) Stackelberg equilibrium condition The Stackelberg game equilibrium must satisfy the following two-level optimization conditions: Upper-level issues (dispatch and control center): (41) Make: (42) Lower-level issues (energy aggregators and load aggregators): Optimal response from energy aggregators: (43) Optimal response of load aggregator: (44) in, , , These represent the optimal strategies for each subject under equilibrium conditions. , These are the revenue functions for the energy aggregator and the load aggregator, respectively. The objective function of the upper-level problem contains the optimal response function of the lower-level problem. and This means that when setting prices, the dispatch control center needs to anticipate the responses of energy aggregators and load aggregators to these prices.

[0075] (2) Algorithm convergence analysis The convergence of the H-MADDPG algorithm mainly depends on the following conditions: Condition 1: Function Approximation Ability: When the Actor network and Critic network have sufficient expressive power, they can approximate the optimal policy function μ and the optimal value function Q with arbitrary precision. Specific network parameters exist. and , so that: (45) in This is the upper bound of the function approximation error.

[0076] Condition 2: Learning Rate Selection: According to the Robbins-Monro stochastic approximation theorem, when the learning rate sequence... The algorithm can converge to a local optimum when the following conditions are met: (46) (1) Ensure that the algorithm has a sufficient step size to reach the vicinity of the optimal solution, and (2) Ensure that the algorithm can eventually stabilize rather than continue to oscillate.

[0077] Condition 3: Exploration noise attenuation: Assume exploration noise satisfy: (47) (1) and (2) ensure that H-MADDPG has sufficient exploration in the early stages of learning and converges to a deterministic policy in the later stages. When the above conditions are met, the H-MADDPG algorithm can converge to the Stackelberg equilibrium with probability 1. Neighborhood: (48) in This represents the Stackelberg equilibrium strategy. , This is the final learning rate.

[0078] (3) Implementation Example Verification This embodiment is validated based on actual data from a virtual power plant in Xiong'an New Area. The system includes 2×300MW thermal power units, a 400MW wind farm, and a 300MW photovoltaic power station. The H-MADDPG algorithm parameters are set as follows: the learning rate of the Actor network is set to... The learning rate of the Critic network is set to Discount factor The soft update coefficient of the target network is set to 0.99 to balance current and future rewards. Set to 0.001 to ensure training stability; the experience replay pool capacity is... It is used to store experiential data generated by the interaction between the agent and the environment, and randomly samples the batch size from it each time the network is updated. The samples are used for training; in the hierarchical update mechanism, the upper-layer DCC agent is trained on samples; The policy is updated once every time step, while the lower-level EA and LA agents update it every [time step]. A strategy update is performed every time step to accurately reflect the difference in decision-making pace between leaders and followers in the Stackelberg game.

[0079] By implementing the method of this invention, system operating costs were reduced by 22%, daily net profit increased by 35%, carbon emissions decreased by 18.3% (from 471 kg / day to 385 kg / day), renewable energy utilization rate increased by 47.3% (wind power utilization rate reached 94%, photovoltaic utilization rate reached 95%), average user satisfaction increased from 0.70 to 0.85, and electricity price fluctuation standard deviation decreased by 37.1% (from 0.35 to 0.22). The H-MADDPG algorithm reached a stable convergence state within approximately 600 rounds, with a convergence speed 25% faster than the traditional MADDPG algorithm, fully verifying the effectiveness and practicality of the method of this invention.

[0080] like Figure 3 As shown, comparing the four optimization schemes (Scheme 1: independent decision-making by each subject; Scheme 2: implementation of demand response; Scheme 3: implementation of cooperative game; Scheme 4: the two-layer game model of this invention), from Scheme 1 to Scheme 4, the system has been significantly improved in terms of economy, environment and renewable energy consumption.

[0081] like Figure 4 As shown, the optimized electricity price signal curve exhibits significantly lower fluctuations compared to the benchmark electricity price, with particularly outstanding control effects during peak electricity consumption periods (09:00-12:00 and 15:00-18:00).

[0082] like Figure 5 As shown, by optimizing the demand response strategy, the overall user satisfaction increased from 0.70 to 0.85, and remained above 0.80 during high-load periods.

[0083] like Figure 6 As shown, the H-MADDPG algorithm converges faster, has smaller oscillation amplitude, and exhibits better stability compared to traditional algorithms.

[0084] Through the above technical solutions, this invention constructs a two-layer game-theoretic collaborative optimization model for a virtual power plant, comprising a dispatch control center, energy aggregators, and load aggregators. The upper layer is the dispatch control center, which, as the leader in the Stackelberg game, guides lower-level decisions by setting dynamic electricity price signals and is responsible for interacting with the external power grid to maintain system power balance. The lower layer includes two followers: one is the energy aggregator, which establishes a cooperative game model to collaboratively optimize supply-side resources such as thermal power units, wind power units, and photovoltaic power generation systems, and uses the Shapley value method for fair profit distribution; the other... The load aggregator constructs a demand response model based on price elasticity theory. While pursuing economic benefits, it takes user satisfaction as a core constraint to ensure the sustainability of the demand response strategy. To efficiently solve this complex two-level game equilibrium problem, an improved hierarchical multi-agent deep deterministic policy gradient (H-MADDPG) reinforcement learning algorithm is proposed. This algorithm reflects the hierarchical relationship of the Stackelberg game by setting different policy network update frequencies for the leader and followers, and embeds Shapley value calculation into the solution process, thereby achieving fast and stable solution of the equilibrium of the entire model.

[0085] This invention demonstrates significant advantages in terms of technological innovation, implementation effectiveness, and application prospects. In terms of technological innovation, this invention is the first to organically combine Stackelberg game theory with cooperative game theory, accurately characterizing the hierarchical decision-making structure and collaborative relationships within a virtual power plant, solving the problem that traditional single-game models cannot fully reflect the complex interactions of multiple stakeholders. The proposed H-MADDPG algorithm reflects the decision-making hierarchy through a hierarchical update mechanism and achieves fair allocation through Shapley value embedding, significantly improving solution efficiency. Compared to traditional algorithms, it increases convergence speed by 25% and reduces computational complexity by 30%. Simultaneously, it achieves multi-objective synergistic optimization of economic benefits, user satisfaction, renewable energy consumption, and carbon emission reduction, overcoming the limitations of single-objective optimization.

[0086] Example 2 Embodiment 2 of the present invention also provides a system for performing the method of Embodiment 1, comprising: The game framework building module is used to establish a two-layer game framework for a virtual power plant, which includes a dispatch control center, energy aggregators, and load aggregators. In this framework, the dispatch control center is the leader, the energy aggregator is the follower, and the load aggregator is the cooperative game participant. The first model building module is used to build the load aggregator optimization model; The second model building module is used to build an energy aggregator optimization model. The third model building module is used to build the optimization model of the scheduling and control center. The optimization solution module is used to adjust the parameters of the load aggregator optimization model, the energy aggregator optimization model, and the dispatch control center optimization model so that each model achieves the optimization objective while meeting its corresponding constraints. The parameters of each model obtained after achieving the optimization objective are used to optimize the control of the actual power plant. The adjusted parameters of the energy aggregator optimization model are the active power output of thermal power units, wind power units, and photovoltaic systems.

[0087] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A virtual power plant collaborative operation optimization method based on two-level game theory, characterized in that, include: S1. Establish a two-layer game framework for a virtual power plant that includes a dispatch control center, energy aggregators, and load aggregators. The dispatch control center is the leader, the energy aggregator is the follower, and the load aggregator is the cooperative game participant. S2. Establish a load aggregator optimization model; S3. Establish an energy aggregator optimization model; S4. Establish an optimization model for the scheduling and control center; S5. Adjust the parameters of the load aggregator optimization model, the energy aggregator optimization model, and the dispatch control center optimization model so that each model achieves the optimization objective under the premise of satisfying its corresponding constraints; use the parameters of each model obtained after achieving the optimization objective to optimize the control of the actual power plant. The adjusted parameters of the energy aggregator optimization model are the active power output of thermal power units, wind power units, and photovoltaic systems.

2. The virtual power plant collaborative operation optimization method based on two-layer game theory according to claim 1, characterized in that, S1 includes: Define the elements of a game ,in For the participants to gather, These represent three decision-making entities: the dispatch and control center, the energy aggregator, and the load aggregator, respectively. The strategy space contains the set of decision variables for each participant. To optimize the objective function set, each participant's strategy is mapped to its payoff value; the strategy space of each participant is defined as follows: in, This represents the price decision vector of the dispatch control center, which includes electricity price settings for 24 time periods. Time periods The transaction price set by the dispatch control center for thermal power units, wind power units, photovoltaic systems and load aggregators; This represents the output decision vector of the energy aggregator; They represent time periods respectively. Active power output of thermal power units, wind power units, and photovoltaic systems This represents the response decision vector of the load aggregator; Indicates time period The demand response price that load aggregators offer to users.

3. The virtual power plant collaborative operation optimization method based on two-layer game theory according to claim 2, characterized in that, S2 includes: The constraints and objective function of the load aggregator are set, and the two together constitute the load aggregator optimization model. The constraints of the load aggregator include... ,in, The minimum satisfaction threshold, Representative time period User overall satisfaction index and , and These are respectively the load reduction satisfaction index and the economic benefit satisfaction index. and These are the weighting coefficients for load reduction satisfaction and economic benefit satisfaction, respectively, to meet... ; The objective function of the load aggregator is: , This represents the comprehensive objective function value of the load aggregator; This is the economic benefit weighting coefficient. The user satisfaction weighting coefficient satisfies , Let be the economic return function. The user satisfaction function is shown in the following formula. in, For time period The electricity price that load aggregators sell to users; For time period The actual electricity sales volume; For time period The electricity purchase price that load aggregators obtain from the dispatch control center; For time period Load following demand response implementation; The constraints on the load aggregator also include ,in, , These are the lower and upper limits of electricity sales prices, respectively. , These are the lower and upper limits of the load, respectively.

4. The virtual power plant collaborative operation optimization method based on two-layer game theory according to claim 1, characterized in that, S3 includes: Models for thermal power units, wind power units, and photovoltaic systems are set up to collectively constitute an energy aggregator optimization model. This energy aggregator optimization model serves as a grand alliance, defining the grand alliance... ,in Representing thermal power units, wind power units, and photovoltaic systems respectively, for any sub-alliance , characteristic function This indicates the maximum potential revenue that the sub-alliance can obtain: in, For the alliance The characteristic function values; For the alliance The set of strategies; For alliance members The payoff function, The optimization objective function for the Major League Baseball is: in, The strategy set for all power generation units. This represents the maximum total benefit that can be obtained when thermal power units, wind power units, and photovoltaic systems are optimized as a whole; the Shapley value method is used for benefit distribution. in, For alliance members The Shapley value represents the alliance member. The distribution of profits due; For the alliance Number of members; The total number of members in the major leagues. To exclude alliance members The characteristic function values ​​of the alliance; The factorial symbol, For combined weights.

5. The virtual power plant collaborative operation optimization method based on two-layer game theory according to claim 4, characterized in that, The thermal power unit model includes: The power generation cost function of a thermal power unit is: in, For time period The power generation cost of thermal power units; For time period The generating capacity of thermal power units This is a secondary cost coefficient; This is the primary cost coefficient; This is the fixed cost coefficient; This refers to the start-up and shutdown cost coefficient. For time period The unit start-stop status variables, Indicates that the unit is in operation. This indicates that the unit is shut down; the relationship between carbon emissions and power generation of a thermal power unit is as follows: in, For time period Carbon emissions; This represents the number of types of greenhouse gases emitted during the power generation process; For the first The fixed emission coefficients of various greenhouse gases, For the first Variable emission factors for various greenhouse gases; thermal power units must meet the following technical constraints: in, To achieve the minimum technical output of thermal power units; To contribute the maximum technical strength to thermal power units; To limit the rate of ascent; To limit the rate of ascent; the revenue function of the thermal power unit is: in, The daily profit of thermal power units; For time period On-grid electricity price for thermal power units; The cost of fuel per unit of electricity generation; The cost of carbon emissions is calculated based on carbon trading prices and actual emissions.

6. The virtual power plant collaborative operation optimization method based on two-layer game theory according to claim 4, characterized in that, The wind turbine model includes: The relationship between wind power and wind speed is described by a power curve: in, For time period Wind turbines at wind speed Power generation under the given conditions; For time period The actual wind speed; To cut in wind speed; Rated wind speed; Cut off the wind speed; This refers to the rated power of the wind turbine generator set; The revenue function of a wind turbine is: in, For the expected profit of wind turbine units, Indicates a scene, For all scene sets; For the scene The probability of; For time period The feed-in tariff for wind power, For time period The power generation capacity of wind turbine units; To calculate the decommissioning benefits over the scheduling cycle; For operation and maintenance costs; The cost is penalized for power deviation.

7. The virtual power plant collaborative operation optimization method based on two-level game theory according to claim 4, characterized in that, The photovoltaic system model includes: Photovoltaic power generation depends on sunlight intensity and ambient temperature: in, For time period The power generation capacity of the photovoltaic system; The conversion efficiency of photovoltaic modules; This refers to the total area of ​​the photovoltaic panels; For time period Light intensity; For temperature coefficient, For time period Ambient temperature; For reference temperature; The payoff function for a photovoltaic system is: in, Expected profits of photovoltaic units Indicates a scene, For all scene sets; For the scene The probability of; For time period The feed-in tariff for photovoltaic systems; To calculate the decommissioning benefits over the scheduling cycle; For operation and maintenance costs; The cost is penalized for power deviation.

8. The virtual power plant collaborative operation optimization method based on two-layer game theory according to claim 4, characterized in that, S4 includes: The constraints and objective function of the scheduling control center are set, and the two together constitute the optimization model of the scheduling control center. The objective function of the scheduling control center is: ,in, For the daily revenue of the dispatch and control center; For time period Electricity sales revenue and , For time period The electricity price sold to load aggregators, For time period Load demand; For time period The cost of purchasing electricity and , , , Time periods The purchase price of electricity from thermal power, wind power, and solar power. , , For the output of each power generation unit; For time period The interaction cost with the external power grid and , For time period Electricity trading prices with external power grids; For time period Interaction power with the external power grid; This is the network loss coefficient; For time period Total output of energy aggregators; The constraints of the dispatch control center include price constraints, power balance constraints, and grid interaction capacity constraints; among which the price constraint is: In the formula, For time period The lower limit of the price for purchasing electricity from external power grids; For time period Price ceiling for selling electricity to external power grids; Power balance constraint is Grid interaction capacity constraints are In the formula, This is the maximum transmission capacity of the connection line to the external power grid.

9. The virtual power plant collaborative operation optimization method based on two-layer game theory according to claim 1, characterized in that, S5 includes: The dispatch control center, energy aggregator, and load aggregator are modeled as three interacting agents. An Actor-Critic network architecture is set up for each agent. The Actor network is used to generate corresponding policy outputs based on the current environmental state; the policy is the agent's parameter adjustment scheme. The Critic network is used to evaluate the Q-value corresponding to the state-policy pair, i.e., the optimization objective function value. In each training iteration, the frequency is... Update the parameters in the scheduling control center, in terms of frequency. Update the parameters of the energy aggregator and load aggregator. The process continues to optimize and update until each agent achieves its optimization goal while satisfying its corresponding constraints.

10. A system for implementing the virtual power plant collaborative operation optimization method based on two-layer game theory as described in any one of claims 1-9, characterized in that, include: The game framework building module is used to establish a two-layer game framework for a virtual power plant, which includes a dispatch control center, energy aggregators, and load aggregators. In this framework, the dispatch control center is the leader, the energy aggregator is the follower, and the load aggregator is the cooperative game participant. The first model building module is used to build the load aggregator optimization model; The second model building module is used to build an energy aggregator optimization model. The third model building module is used to build the optimization model of the scheduling and control center. The optimization solution module is used to adjust the parameters of the load aggregator optimization model, the energy aggregator optimization model, and the dispatch control center optimization model so that each model achieves the optimization objective while meeting its corresponding constraints. The parameters of each model obtained after achieving the optimization objective are used to optimize the control of the actual power plant. The adjusted parameters of the energy aggregator optimization model are the active power output of thermal power units, wind power units, and photovoltaic systems.

Images