A multi-agent game collaborative planning method for airport comprehensive energy system under physical-information-market coupling
By introducing dynamic game theory and multi-agent reinforcement learning algorithms into the airport integrated energy system, the problems of conflicting interests among multiple stakeholders and coupling of short-term and long-term decisions were solved. This enabled collaborative planning among multiple decision-makers, reduced system costs and carbon emissions, and improved the absorption capacity of renewable energy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG UNIV
- Filing Date
- 2026-03-12
- Publication Date
- 2026-06-26
AI Technical Summary
Traditional energy system planning methods ignore the independent decision-making rights and interests of different stakeholders, leading to conflicts of interest and making it difficult to coordinate and optimize short-term and long-term decisions within a unified framework, thus affecting the implementation of planning schemes.
By employing dynamic game theory and multi-agent reinforcement learning algorithms, a multi-agent game collaborative planning method is constructed. Through the Stackelberg game framework and multi-agent reinforcement learning algorithms, the short-term and long-term decisions of multiple decision-makers in the airport integrated energy system are optimized, thereby achieving coordination of conflicting interests and unification of decisions.
It significantly reduced the difficulties in implementing plans due to conflicts of interest, optimized the integration of short-term and long-term decisions, reduced the total system cost and carbon emissions, and improved the capacity to absorb renewable energy.
Smart Images
Figure CN122288198A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of integrated energy systems and smart grids, specifically relating to a multi-stakeholder game-based collaborative planning method for airport integrated energy systems under physical-information-market coupling. Background Technology
[0002] The construction of new power systems based on renewable energy is accelerating, and integrated energy systems (IES), as a key technology for achieving multi-energy complementarity, improving energy efficiency, and promoting the consumption of renewable energy, has become a research hotspot in the energy field. Especially in energy-intensive public building complexes such as airports, high-tech parks, and large commercial complexes, the planning and construction of IES is of great significance for regional energy conservation and emission reduction.
[0003] However, traditional energy system planning methods typically employ centralized optimization, treating the entire system as a single entity and setting a unified planning objective. This approach ignores the fact that different stakeholders (such as the power grid, energy service providers, users, and independent investors) possess independent decision-making power and interests. For example, smart grids focus on grid security and stability and return on investment; integrated airport energy systems aim to minimize their own operating costs; and renewable energy companies pursue profit maximization for their investment projects. These conflicts of interest among stakeholders make traditional planning schemes difficult to implement in practice.
[0004] Furthermore, energy system planning is a complex decision-making process involving long-term investment and short-term operation. Long-term investments (such as the site selection and capacity determination of generator units and energy storage equipment) are closely coupled with short-term operations (such as daily power generation scheduling and energy trading). Current investment decisions profoundly affect future operational efficiency and flexibility, and conversely, future operational expectations determine the value of current investments. How to coordinate and optimize long-term and short-term decisions within a unified framework to avoid investment redundancy or short-sightedness is another major challenge facing current technologies.
[0005] To address the aforementioned challenges, a key technical problem urgently needs to be solved: how to construct a comprehensive energy system planning methodology that can accurately reflect the interactive behavior of multiple stakeholders in the market and effectively coordinate short- and long-term decisions. Summary of the Invention
[0006] The purpose of this invention is to overcome the limitations of existing technologies and provide a multi-stage collaborative planning method for integrated energy systems and smart grids based on multi-agent dynamic game theory. This method aims to solve the problems of multi-agent conflict of interest and coupling of long-term and short-term decisions in the planning of large-scale integrated energy systems such as airports. By introducing advanced algorithms such as dynamic game theory and multi-agent reinforcement learning, it constructs an intelligent planning and control system that can adapt to dynamic markets and complex operating environments. This invention particularly focuses on how to consider the long-term investment and short-term operational decisions of multiple decision-making entities, such as smart grids, multiple airport integrated energy systems, renewable energy investors, and load aggregators, and utilizes dynamic game theory to resolve the conflict of interest and cooperation issues among these entities in the multi-stage planning process.
[0007] To achieve the above objectives, the present invention provides the following technical solution: A multi-agent game-theoretic collaborative planning method for an airport integrated energy system under physical-information-market coupling includes the following steps: S1 defines the multiple decision-making entities involved in the planning, including the airport integrated energy system as the hub, smart grid, n Each load aggregator and renewable energy operator is an independent decision-making entity; the system set and index are defined, and the overall planning period is set. The overall planning period K is divided into multiple phases. k Establish a multi-phase planning state transition process model; clarify the roles of each decision-making body in each phase. The set of decision variables; S2, Construct a physical-information-market coupling model for the interaction between the integrated energy system and the smart grid; S3, based on the physical-information-market coupling model constructed in S2, sets the subgame perfect Nash equilibrium as the solution objective, and constructs a system with the smart grid as the leader, and the airport integrated energy system, renewable energy operators, and... n The Stackelberg game equilibrium problem in which the load aggregator is a follower; S4. By constructing a multi-agent reinforcement learning algorithm, the Stackelberg game equilibrium problem constructed in S3 is solved to obtain the Stackelberg game equilibrium strategy. S5, based on the Stackelberg game equilibrium strategy obtained in S4, generates a multi-stage equipment investment time series diagram under multi-entity collaboration and outputs a typical daily operation scheduling scheme, and evaluates it.
[0008] Furthermore, step S1 includes the following steps: The set and index of the definition system include: a set of decision-making entities. , IES For the airport's integrated energy system, For smart grids, For the i-th load aggregation quotient, For renewable energy operators; operating time within a single phase set , ; The multi-stage planning state transition process model is specifically as follows: each stage is coupled using equipment capacity as a state variable; each decision-making entity... i In the previous stage decision variable set and the previous stage Initial equipment capacity The current stage has been determined. Initial equipment capacity ; Various decision-making bodies i In the stage decision variable set Includes: Airport Integrated Energy System (IES) decision variable set Smart grid decision variable set renewable energy operators decision variable set Load aggregation merchants decision variable set Each independent decision-making entity's set of decision variables includes both long-term investment and short-term operations.
[0009] Furthermore, the airport integrated energy system IES decision variable set Includes: Airport Integrated Energy System (IES) In the stage New investment capacity Airport Integrated Energy System (IES) In the stage Equipment scheduling output Airport Integrated Energy System (IES) In the stage Electricity purchase from smart grid ; The smart grid decision variable set Including: power grid expansion investment Pricing of power transmission services ; The renewable energy operator decision variable set Including: Renewable energy operators In the stage Power plant site selection decision Renewable energy operators In the stage Power plant capacity decision Renewable energy operators In the stage k The price of selling electricity to the smart grid Smart grids are shifting towards renewable energy operators in phase k. RE Electricity purchase ; The load aggregator decision variable set include: Load aggregation In the stage Electricity purchased from smart grid ; Load aggregation In the stage Power participating in smart grid demand response regulation Among them, the power regulation associated with load aggregation quotient In the stage Economic compensation for participating in smart grid demand response regulation .
[0010] Furthermore, step S2 includes the following steps: The physical-information-market coupling model of the interaction between the integrated energy system and the smart grid Including physical layer Market level and Information Layer Inf: The physical layer The operating laws of smart grids, combined heat and power, gas boilers, energy storage, and photovoltaic / wind power are represented by energy conservation and operating constraints, including power balance P1, thermal balance P2, combined heat and power unit constraints P3, gas boiler constraints P4, and energy storage constraints P5. The market layer Representing all participating decision-making entities i The objective function includes: airport integrated energy system The goal is to minimize total energy costs. Smart grid The goal is to maximize net present value. renewable energy operators RE The goal is to maximize net present value. Load aggregation merchants The goal is to minimize energy costs. ; The information layer Inf includes stage k smart grid G transmission service pricing and stage k renewable energy operators. The price and stage of selling electricity to the smart grid G Load aggregation Economic compensation for participation in smart grid demand response regulation and carbon tax prices at stage k. The physical-information-market coupling specifically refers to the coupling of electricity prices and the coupling of electricity purchase volume, which together constitute the smart grid. , n Individual load aggregator renewable energy operators RE Multi-party game between them; The electricity price coupling refers to the electricity price set by the smart grid. These are decision variables for smart grids, and are relevant to airport integrated energy systems. and load aggregation merchants These are external cost parameters that affect the electricity purchase costs of the airport's integrated energy system and load aggregators; The electricity purchase coupling refers to: the electricity purchase decision of each airport's integrated energy system. It is its own physical layer decision variable, while the airport integrated energy system and load aggregation merchants The total amount of electricity purchased constitutes the revenue source of the smart grid at the market level.
[0011] Furthermore, the power balance P1 is represented as: ; In the formula: A collection of power generation equipment, including combined heat and power units and photovoltaic / wind power; For equipment At any moment The electrical power output; For the smart grid at stage k time to renewable energy operators RE Electricity purchase; For energy storage devices at time k The discharge power; For load aggregation i At stage k The electricity load demand; For energy storage devices at time k The charging power; Integrated Energy Systems for Airports (IES) In the stage time t Electricity purchased from the smart grid; The thermal balance P2 is expressed as: ; In the formula: It is a collection of heat-generating equipment, including combined heat and power units and gas-fired boilers; For heat generation equipment j In the stage k time The heat production capacity is as follows; For load aggregation i In the stage k time Thermal power output; Integrated Energy Systems for Airports (IES) In the stage k time Heat load requirements; The constraint P3 of the combined heat and power unit is represented as follows: ; in, For cogeneration units in the stage k time Thermal power; For cogeneration units in the stage k time The electrical power; For cogeneration units in the stage The installed capacity; Thermoelectric ratio; The constraint P4 of the gas-fired boiler is represented as follows: ; in, Integrated Energy Systems for Airports (IES) China Gas Boiler in Stage k time The heating capacity; Integrated Energy Systems for Airports (IES) In the stage The installed capacity of gas-fired boilers; For gas-fired boiler efficiency; Integrated Energy Systems for Airports (IES) China Gas Boiler in Stage k time Gas consumption; The energy storage constraint P5 is expressed as: ; in, and For energy storage devices at different times +1 and time t Energy reserves; The charging efficiency at stage k; The discharge efficiency of the energy storage device in stage k; For energy storage devices at time k t The charging power; For energy storage devices at time k t The discharge power; This represents the maximum capacity of the energy storage device in stage k.
[0012] Furthermore, the airport integrated energy system The goal is to minimize total energy costs. Its objective function is as follows: ; in, The discount rate is... Airport Integrated Energy System In the stage k The investment cost, Airport Integrated Energy System Investment in stage k, Airport Integrated Energy System The operational costs at stage k Airport Integrated Energy System Energy purchase cost at stage k Airport Integrated Energy System In the stage k Revenue from electricity sales Airport Integrated Energy System In the stage k The carbon emission costs; For the stage k The price of carbon tax; The smart grid The goal is to maximize net present value. Its objective function is as follows: in, For smart grid Electricity sales revenue, For smart grid Investment costs, For smart grid Operation and maintenance costs; The renewable energy operator RE The goal is to maximize net present value. Its objective function is as follows: in, For renewable energy operators REElectricity sales revenue, For renewable energy operators RE Investment costs, For renewable energy operators RE Operation and maintenance costs; The load aggregator The goal is to minimize energy costs. Its objective function is as follows: .
[0013] Furthermore, step S3 includes the following steps: The Stackelberg game equilibrium problem is formulated as a two-level optimization problem, as follows: The upper-level optimization problem, with the smart grid G as the leader: ; The set of equilibrium decision variables for each follower Simultaneously satisfying the following Nash equilibrium conditions: Lower-level problems: equilibrium problems in multi-follower non-cooperative games, including: For airport integrated energy system : For renewable energy operators : For load aggregator : in, The price at which electricity is sold to the smart grid. For the investment in the airport integrated energy system in stage k, For the interaction power between the airport IES operator and the smart grid in stage k, For renewable energy operators' investments in phase k, The actual output power of renewable energy. The interaction power between the renewable energy operator and the smart grid in stage k is represented by variables marked with an asterisk (*), which indicate the equilibrium strategies of other decision-makers. The optimization problems of the aforementioned decision-making entities must all reach their optimal state, collectively forming the Nash equilibrium solution of the lower-level game. The optimization problem involves feeding back the Nash equilibrium solution as a parameter to the upper-level leader.
[0014] Furthermore, the construction of the multi-agent reinforcement learning algorithm in step S4 includes the following steps: Step S41: Construction of a multi-agent reinforcement learning framework; Each decision-making entity is constructed as an intelligent agent. i ,include , , , n indivual Each intelligent agent i Having a state space Action space Reward function ; Step S42: Training mechanism and algorithm selection; A centralized training and distributed execution framework is adopted. During the training process, the agent acquires global state information, while during the execution phase, it only relies on local observations. The algorithms used include one of the MADDPG, MAPPO, and QMIX algorithms. Step S43: Multi-agent training process; Initialize the policy network and evaluation network of all agents, set the hyperparameters, and create experience replay buffers for each agent to store the state-action-reward-next state tuples during the interaction process; In each training round, each agent selects an action based on the current policy, interacts with other agents in the environment and the physical-information-market coupling model, collects complete state transition samples, and stores them in their respective experience replay buffers; A centralized training method is adopted, in which a batch of samples are sampled from the experience replay buffer and the Critic and Actor networks of each agent are updated respectively; Continuously monitor the policy change magnitude and reward function convergence of each agent; when the action distribution output by the policy network tends to stabilize and the reward function of all agents fluctuates less than the preset threshold in several consecutive rounds, it is considered that the system has converged to the vicinity of Nash equilibrium, and the training process is terminated; complete the construction of the multi-agent reinforcement learning algorithm.
[0015] Furthermore, in step S4, each agent... i Having a state space Action space Reward function Specifically: For smart grids Building intelligent agents state space Action space reward function , For smart grid Investment capacity at stage k; For airport integrated energy system Building intelligent agents state space Action space reward function , Airport Integrated Energy System Investment capacity at stage k; For renewable energy operators Building intelligent agents state space Action space reward function , For renewable energy operators Investment capacity at stage k; for n Individual load aggregator Construct n intelligent agents state space Action space reward function .
[0016] Furthermore, step S5 includes the following steps: Step S51: Generate optimal decision variables for multiple decision-makers. Based on the policy network of each agent after training convergence, the output of each decision-making agent at each planning stage is provided. The optimal decision variables include: smart grid Optimal transmission service pricing and optimal power grid expansion investment Airport Integrated Energy System Optimal equipment investment capacity Optimal scheduling output and optimal purchase volume Renewable energy operators Optimal power plant site selection decision Optimal power plant capacity decision renewable energy operators RE Optimal price for selling electricity to the smart grid and smart grid G to renewable energy operators RE Optimal purchase volume ; Various load aggregation merchants Optimal electricity purchase from smart grid and optimal power for participation in smart grid demand response regulation ; Step S52, output the typical daily scheduling scheme: For typical daily / seasonal operating scenarios, the optimal decision variables of each decision-making entity are generated based on S51 for each hour. The scheduling schemes for multi-energy production, storage, trading, and consumption include: real-time power and heat balance schemes; charging and discharging plans for energy storage devices; energy interaction plans with the power grid; demand response participation and economic compensation settlement; Step S53, Comprehensive evaluation of the planning scheme: The generated planning scheme is evaluated from three dimensions: economic efficiency, environmental friendliness, and system reliability. Specifically, this includes: Economic assessment: Calculate the total system cost, net present value of each entity, and investment payback period indicators; Environmental assessment: Calculate the system's total carbon emissions, renewable energy penetration rate, and carbon emission reductions; Sensitivity analysis: Analyzing the impact of key parameters on the optimal decision variables; Finally, the output includes a timeline of equipment investment for multiple periods, a scheduling scheme for typical operating days, economic and environmental assessment results, and sensitivity analysis results.
[0017] The beneficial effects of this invention are: (1) This invention constructs a Stackelberg game framework with smart grid as the leader and multiple energy operators as followers, and uses a multi-agent reinforcement learning algorithm to solve the equilibrium strategy, thereby realizing the collaboration of multiple independent decision-making entities at the investment and operation level, and significantly reducing the problem of planning implementation difficulties caused by conflict of interest.
[0018] (2) This invention integrates long-term equipment investment with short-term scheduling and operation through a multi-period planning state transition process model, optimizes decision-making in a unified framework, avoids redundant investment, and improves operating efficiency. It aims to reduce the total system cost and carbon emissions and enhance the renewable energy absorption capacity.
[0019] (3) This invention can generate clear multi-phase equipment investment sequences, typical daily scheduling strategies and multi-dimensional evaluation results, providing quantitative basis for the actual planning and construction of integrated energy systems in large parks such as airports, and providing scientific analysis tools for market mechanisms. Attached Figure Description
[0020] Figure 1 This is a flowchart of a multi-agent game-theoretic collaborative planning method for an airport integrated energy system under physical-information-market coupling, as described in this invention.
[0021] Figure 2This is a schematic diagram of the multi-agent game in the multi-agent game collaborative planning method for an airport integrated energy system under physical-information-market coupling as described in this invention.
[0022] Figure 3 This is the MADDPG training convergence curve in the multi-agent game collaborative planning method for an airport integrated energy system under physical-information-market coupling as described in this invention.
[0023] Figure 4 This invention compares the performance of multi-agent Nash equilibrium in a multi-agent game-theoretic collaborative planning method for an airport integrated energy system under physical-information-market coupling.
[0024] Figure 5 This is a time series diagram of multi-period equipment investment in a multi-agent game collaborative planning method for an airport integrated energy system under physical-information-market coupling as described in this invention.
[0025] Figure 6 This is a typical daily scheduling scheme in the multi-agent game collaborative planning method for an airport integrated energy system under physical-information-market coupling described in this invention.
[0026] Figure 7 This is the economic evaluation result of a multi-agent game-based collaborative planning method for an airport integrated energy system under physical-information-market coupling as described in this invention.
[0027] Figure 8 This is the environmental assessment result in the multi-agent game collaborative planning method for airport integrated energy system under physical-information-market coupling described in this invention.
[0028] Figure 9 This is the sensitivity analysis result of a multi-agent game-theoretic collaborative planning method for an airport integrated energy system under physical-information-market coupling as described in this invention. Detailed Implementation
[0029] The present invention will be further described below with reference to the accompanying drawings and preferred embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.
[0030] It should be noted that the directional terms such as left, right, up, down, top, and bottom used in this embodiment are only relative concepts or are based on the normal use of the product, and should not be considered as restrictive.
[0031] Example: like Figure 1 This is a schematic diagram of a multi-agent game-theoretic collaborative planning method for an airport integrated energy system under physical-information-market coupling, according to the present invention. Figure 2This is a schematic diagram of the multi-agent game in the multi-agent game collaborative planning method for an airport integrated energy system under physical-information-market coupling described in this invention. Large civil airports are typical public buildings. Taking an integrated energy system of a large airport building located in a frigid region as an example, this invention provides a detailed description of the method, which includes the following steps: S1 defines the multiple decision-making entities involved in the planning. i Specifically, this includes airport integrated energy systems. As a hub, define the smart grid , n Individual load aggregator renewable energy operators RE As an independent decision-making entity; define the system's set and index, and set the overall planning period K; divide the planning period K into multiple stages. k Establish a multi-phase planning state transition process Clearly define the decision-making bodies. i In the stage decision variable set ; S2, Construct a physical-information-market coupling model for the interaction between the integrated energy system and the smart grid. ; S3, based on the physical-information-market coupling model constructed in S2 The goal is to solve for the subgame perfect Nash equilibrium, and a system is constructed with the smart grid G as the leader and the airport integrated energy system as the supporting system. renewable energy operators as well as n Individual load aggregator The Stackelberg game equilibrium problem for followers; S4, by constructing a multi-agent reinforcement learning algorithm, solve the Stackelberg game equilibrium problem constructed in S3, and obtain the Stackelberg game equilibrium strategy. Specifically, it includes S41 construction of the multi-agent reinforcement learning framework, S42 training mechanism and algorithm selection, and S43 multi-agent training process. S5, based on the Stackelberg game equilibrium strategy obtained in S4, generates a multi-stage equipment investment time series diagram under multi-entity collaboration and outputs a typical daily operation scheduling scheme, and evaluates it.
[0032] Furthermore, in step S1: The definition of multiple decision-making entities involved in the planning i Specifically, this includes airport integrated energy systems. As a hub, define the smart grid 1 load aggregator i=1, renewable energy operator RE It is an independent decision-making body.
[0033] The set and index of the definition system specifically include: The collection of decision-making bodies , ; Runtime within a single stage set , ; Set the overall planning period Divided into multiple stages , .
[0034] Furthermore, the planning period K is divided into multiple stages. k Establish a multi-stage planning state transition process model; Each stage is coupled through the state variable of equipment capacity. Multiple entities are involved in the previous stage. Decision variables The current stage has been determined. Initial equipment capacity The state transition process can be represented as: This state transition process links long-term investment decisions at each stage, forming the basis of dynamic game theory.
[0035] Furthermore, it clarifies the roles of multiple stakeholders in each stage. decision variable set This mainly includes the Airport Integrated Energy System (IES). decision variable set Smart grid decision variable set renewable energy operators decision variable set .
[0036] During the planning stage In this context, each independent decision-making entity has its own set of decision variables, including both long-term investment and short-term operations.
[0037] Airport Integrated Energy System (IES) decision variable set It can be represented as: in: Integrated Energy Systems for Airports (IES) In the stage The new investment capacity; Integrated Energy Systems for Airports (IES) In the stage Equipment scheduling output; Integrated Energy Systems for Airports (IES) In the stage From smart grid Electricity purchased.
[0038] Smart Grid decision variable set It can be represented as: in, Investment to expand the capacity of the smart grid in phase k; Pricing for the G-stage smart grid transmission service.
[0039] renewable energy operators decision variable set It can be represented as: Among them: Site For renewable energy operators In the stage Power plant site selection decision; For renewable energy operators In the stage Power plant capacity decision; For renewable energy operators The price at which electricity is sold to the smart grid in stage k; For smart grids in phase k to renewable energy operators RE Electricity purchased.
[0040] Load aggregation decision variable set It can be represented as: in: For load aggregation In the stage Purchase electricity from the smart grid; For load aggregation In the stage Power involved in demand response regulation of smart grids; For load aggregation In the stage Economic compensation for participation in smart grid demand response regulation.
[0041] Furthermore, in step S2: Construct a physical-information-market coupling model for the interaction between the integrated energy system and the smart grid. Including physical layer Market level and the information layer Inf, represented as : physical layer The core of the physical layer is to express the operating laws of smart grids, combined heat and power, gas-fired boilers, energy storage, and photovoltaic / wind power using energy conservation and operational constraints, which can be represented as follows: ; Market level Economic rules representing the energy trading and policy environment, and the various decision-making entities involved. i The objective function is expressed as: ; Information layer (Inf): Serving as a bridge between the physical layer and the market layer, it transmits market electricity price information through the information system, enabling various decision-making entities to... i The ability to adjust strategies based on market signals is expressed as .
[0042] Furthermore, the physical layer Specifically as follows: The power balance P1 is represented as: A collection of power generation equipment (combined heat and power units, photovoltaic / wind power); For equipment At any moment The electrical power output; For the smart grid at stage k time to renewable energy operators RE Electricity purchase; For energy storage devices at time k The discharge power; For load aggregation i At stage k The electricity load demand; For energy storage devices at time k The charging power; Integrated Energy Systems for Airports (IES) In the stage time t Electricity purchased from the smart grid.
[0043] The thermal equilibrium P2 is expressed as: A collection of heat-generating equipment (combined heat and power units, gas-fired boilers); For load aggregation i In the stage k time Thermal power output; Integrated Energy Systems for Airports (IES) In the stage k time The heat load requirement.
[0044] The constraint P3 of the combined heat and power unit is represented as: in, For cogeneration units in the stage k time Thermal power; For cogeneration units in the stage k time The electrical power; For cogeneration units in the stage The installed capacity; This is the thermoelectric ratio coefficient.
[0045] The constraint P4 for the gas-fired boiler is represented as: in, Integrated Energy Systems for Airports (IES) China Gas Boiler in Stage k time The heating capacity; Integrated Energy Systems for Airports (IES) In the stage The installed capacity of gas-fired boilers; For gas-fired boiler efficiency; Integrated Energy Systems for Airports (IES) China Gas Boiler in Stage k time The amount of gas consumed.
[0046] Energy storage constraint P5 is expressed as: in, and For energy storage devices at different times +1 and time t Energy reserves; The charging efficiency at stage k; The discharge efficiency of the energy storage device in stage k; For energy storage devices at time k t The charging power; For energy storage devices at time kt The discharge power; This represents the maximum capacity of the energy storage device in stage k.
[0047] Furthermore, the objective function for multiple market participants is as follows: 1) Airport Integrated Energy System The goal is to minimize total energy costs. : in, The discount rate is... Airport Integrated Energy System In the stage k The investment cost, Airport Integrated Energy System The operational costs at stage k Airport Integrated Energy System Energy purchase cost at stage k Airport Integrated Energy System Electricity sales revenue at stage k Airport Integrated Energy System The carbon emission cost at stage k; Let k be the carbon tax price at stage k.
[0048] 2) Smart Grid The goal is to maximize net present value. Its objective function is as follows: in, For smart grid Electricity sales revenue, For smart grid Investment costs, For smart grid Operation and maintenance costs.
[0049] renewable energy operators RE The goal is to maximize net present value. Its objective function is as follows: in, For renewable energy operators RE Electricity sales revenue, For renewable energy operators RE Investment costs, For renewable energy operators RE Operation and maintenance costs.
[0050] Load aggregation The goal is to minimize energy costs. Its objective function is as follows: Furthermore, the physical-information-market coupling is manifested in the fact that the decision variables of one entity become part of the objective function of another entity. Here, it mainly refers to the coupling between electricity price and electricity purchase volume, which together constitute the smart grid. , n Individual load aggregator renewable energy operators RE A multi-party game between them.
[0051] Furthermore, the aforementioned electricity price coupling refers to the electricity price set by the smart grid. It is its own decision variable, but for the airport integrated energy system and load aggregation merchants For them, this is an external cost parameter that directly affects their electricity purchase cost.
[0052] The term "electricity purchase coupling" refers to the electricity purchase decisions made by each airport's integrated energy system. It is its own physical layer decision variable, while the airport integrated energy system and load aggregation merchants The total amount of electricity purchased constitutes a key source of revenue for the smart grid at the market level.
[0053] Furthermore, in step S3: The Stackelberg game framework in the Stackelberg game equilibrium problem can be expressed as a two-level optimization problem, as follows: The upper-level optimization problem, with the smart grid G as the leader: ; The set of equilibrium decision variables for each follower Simultaneously satisfying the following Nash equilibrium conditions: Lower-level problems: equilibrium problems in multi-follower non-cooperative games, including: For airport integrated energy system : For renewable energy operators : For load aggregator : in, The price at which electricity is sold to the smart grid. For the investment in the airport integrated energy system in stage k, The interaction power between the airport's integrated energy system and the smart grid at stage k. For renewable energy operators' investments in phase k, The actual output power of renewable energy. The interaction power between the renewable energy operator and the smart grid in stage k is represented by variables marked with an asterisk (*), which indicate the equilibrium strategies of other decision-makers. The optimization problems of the aforementioned decision-making entities must all reach their optimal state, collectively forming the Nash equilibrium solution of the lower-level game. The optimization problem involves feeding back the Nash equilibrium solution as a parameter to the upper-level leader.
[0054] Given the leader's (smart grid G) strategy, all followers at the lower level (including the airport integrated energy system) renewable energy operators and 1 load aggregator Simultaneous decision-making creates a non-cooperative game problem. Each follower optimizes its own objective function, and its decisions are influenced by the decisions of other followers. The final state at the lower level is to reach a Nash equilibrium, i.e., to find a set of strategies that are suitable for all followers. This ensures that for each follower, given the equilibrium strategies of other followers and the leader, their strategy is optimal.
[0055] Furthermore, in step S4: Step S41: Construction of a Multi-Agent Reinforcement Learning Framework Each decision-making entity (smart grid) Airport Integrated Energy System renewable energy operators 1 load aggregator Each is constructed as an intelligent agent. i ( , , , n indivual Each intelligent agent i Having a state space Action space Reward function : For smart grids Building intelligent agents , Its state space ; Its action space ; Its reward function , For smart grid Investment capacity at stage k; For airport integrated energy system Building intelligent agents , Its state space ; Its action space ; Its reward function , For airport integrated energy systems; For renewable energy operators Building intelligent agents , Its state space ; Its action space ; Its reward function , For renewable energy operators Investment capacity at stage k; for n Individual load aggregator Construct n intelligent agents , Its state space ; Its action space ; Its reward function ; Step S42: Training Mechanism and Algorithm Selection A centralized training and distributed execution framework is adopted. During training, the agent can acquire global state information, but during the execution phase, it only relies on local observations. Algorithms such as MADDPG, MAPPO, and QMIX can be used; this invention employs the MADDPG algorithm for solving the problem.
[0056] Step S43: Multi-agent training process Initialize the policy network and evaluation network for all agents (including leaders and followers), set the hyperparameters, and create experience replay buffers for each agent to store state-action-reward-next state tuples during the interaction process. ; In each training round, each agent selects an action based on the current policy, interacts with other agents in the environment and the physical-information-market coupling model, collects complete state transition samples, and stores them in their respective experience replay buffers.
[0057] A centralized training method is adopted, in which a batch of samples are sampled from the experience replay buffer and the Critic and Actor networks of each agent are updated respectively.
[0058] The system continuously monitors the magnitude of policy changes and the convergence of reward functions for each agent. When the action distribution output by the policy network tends to stabilize, and the fluctuations of the reward functions of all agents are less than a preset threshold over several consecutive rounds, the system is considered to have converged to near Nash equilibrium, and the training process is terminated.
[0059] The convergence of the reward function of the four agents during the training process of the MADDPG algorithm is as follows: Figure 3 As shown, the performance comparison results of multi-agent Nash equilibrium are as follows: Figure 4 As shown.
[0060] Furthermore, in step S5: Step S51: Generate a multi-phase equipment investment timeline diagram: Based on the policy network of each agent after training convergence, the output of each decision-making agent at each planning stage is provided. The optimal decision variables were determined, resulting in a multi-period equipment investment time series diagram, including: smart grid. Optimal transmission service pricing and optimal power grid expansion investment Airport Integrated Energy System Optimal equipment investment capacity Optimal scheduling output and optimal purchase volume Renewable energy operators Optimal power plant site selection decision Optimal power plant capacity decision renewable energy operators RE Optimal price for selling electricity to the smart grid and smart grid G to renewable energy operators RE Optimal purchase volume ; Various load aggregation merchants Optimal electricity purchase from smart grid and optimal power for participation in smart grid demand response regulation ; Step S52, output the typical daily scheduling scheme: For a typical daily load, the optimal decision variables based on S51 are generated for each decision-making entity at the hourly level. A detailed scheduling scheme for multi-energy production, storage, trading, and consumption, including: Real-time balance schemes for electricity and heat; charging and discharging plans for energy storage devices; energy interaction plans with the power grid; demand response participation and economic compensation settlement.
[0061] Step S53, Comprehensive evaluation of the planning scheme: The generated planning scheme is evaluated from three dimensions: economic efficiency, environmental friendliness, and system reliability. Specifically, this includes: Economic assessment: Calculate the total system cost, net present value of each entity, and investment payback period indicators; Environmental assessment: Estimate the system's total carbon emissions, renewable energy penetration rate, and carbon emission reductions; Sensitivity analysis: The impact of key parameters on the optimal decision variables is analyzed. In this embodiment, the key parameters are selected as electricity price, carbon price, natural gas price, and renewable energy investment cost.
[0062] Finally, the output includes the timeline of equipment investment in multiple periods, such as... Figure 5 As shown, a typical daily scheduling scheme is as follows: Figure 6 As shown, the economic evaluation results are as follows: Figure 7 As shown, the environmental assessment results are as follows: Figure 8 As shown, the sensitivity analysis results are as follows: Figure 9 As shown.
Claims
1. A multi-agent game-theoretic collaborative planning method for an airport integrated energy system under physical-information-market coupling, characterized in that, Includes the following steps: S1 defines the multiple decision-making entities involved in the planning, including the airport integrated energy system as the hub, smart grid, n Each load aggregator and renewable energy operator is an independent decision-making entity; the system set and index are defined, and the overall planning period K is set; the overall planning period K is divided into multiple stages. k Establish a multi-phase planning state transition process model; clarify the roles of each decision-making body in each phase. The set of decision variables; S2, Construct a physical-information-market coupling model for the interaction between the integrated energy system and the smart grid; S3, based on the physical-information-market coupling model constructed in S2, sets the subgame perfect Nash equilibrium as the solution objective, and constructs a system with the smart grid as the leader, and the airport integrated energy system, renewable energy operators, and... n The Stackelberg game equilibrium problem in which the load aggregator is a follower; S4. By constructing a multi-agent reinforcement learning algorithm, the Stackelberg game equilibrium problem constructed in S3 is solved to obtain the Stackelberg game equilibrium strategy. S5, based on the Stackelberg game equilibrium strategy obtained in S4, generates a multi-stage equipment investment time series diagram under multi-entity collaboration and outputs a typical daily operation scheduling scheme, and evaluates it.
2. The multi-agent game-theoretic collaborative planning method for an airport integrated energy system under physical-information-market coupling as described in claim 1, characterized in that, Step S1 includes the following steps: The set and index of the definition system include: a set of decision-making entities. , IES For the airport's integrated energy system, For smart grids, For the i-th load aggregation quotient, For renewable energy operators; operating time within a single phase set , ; The multi-stage planning state transition process model is specifically as follows: each stage is coupled using equipment capacity as a state variable; each decision-making entity... i In the previous stage decision variable set and the previous stage Initial equipment capacity The current stage has been determined. Initial equipment capacity ; Various decision-making bodies i In the stage decision variable set Includes: Airport Integrated Energy System (IES) decision variable set Smart grid decision variable set renewable energy operators decision variable set Load aggregation merchants decision variable set Each independent decision-making entity's set of decision variables includes both long-term investment and short-term operations.
3. The multi-agent game-theoretic collaborative planning method for an airport integrated energy system under physical-information-market coupling as described in claim 2, characterized in that, The airport integrated energy system IES decision variable set Includes: Airport Integrated Energy System (IES) In the stage New investment capacity Airport Integrated Energy System (IES) In the stage Equipment scheduling output Airport Integrated Energy System (IES) In the stage Electricity purchase from smart grid ; The smart grid decision variable set Including: power grid expansion investment Pricing of power transmission services ; The renewable energy operator decision variable set Including: Renewable energy operators In the stage Power plant site selection decision Renewable energy operators In the stage Power plant capacity decision Renewable energy operators In the stage k The price of selling electricity to the smart grid Smart grids are shifting towards renewable energy operators in phase k. RE Electricity purchase ; The load aggregator decision variable set include: Load aggregation In the stage Electricity purchased from smart grid ; Load aggregation In the stage Power participating in smart grid demand response regulation Among them, the power regulation associated with load aggregation quotient In the stage Economic compensation for participating in smart grid demand response regulation .
4. The multi-agent game-theoretic collaborative planning method for an airport integrated energy system under physical-information-market coupling as described in claim 3, characterized in that, Step S2 includes the following steps: The physical-information-market coupling model of the interaction between the integrated energy system and the smart grid Including physical layer Market level and Information Layer Inf: The physical layer The operating laws of smart grids, combined heat and power, gas boilers, energy storage, and photovoltaic / wind power are represented by energy conservation and operating constraints, including power balance P1, thermal balance P2, combined heat and power unit constraints P3, gas boiler constraints P4, and energy storage constraints P5. The market layer Representing all participating decision-making entities i The objective function includes: airport integrated energy system The goal is to minimize total energy costs. Smart grid The goal is to maximize net present value. renewable energy operators RE The goal is to maximize net present value. Load aggregation merchants The goal is to minimize energy costs. ; The information layer Inf includes stage k smart grid G transmission service pricing, stage k renewable energy operators The price and stage of selling electricity to the smart grid G Load aggregation Economic compensation for participation in smart grid demand response regulation and carbon tax prices at stage k. The physical-information-market coupling specifically refers to the coupling of electricity prices and the coupling of electricity purchase volume, which together constitute the smart grid. , n Individual load aggregator renewable energy operators RE Multi-party game between them; The electricity price coupling refers to the electricity price set by the smart grid. These are decision variables for smart grids, and are relevant to airport integrated energy systems. and load aggregation merchants These are external cost parameters that affect the electricity purchase costs of the airport's integrated energy system and load aggregators; The electricity purchase coupling refers to: the electricity purchase decision of each airport's integrated energy system. It is its own physical layer decision variable, while the airport integrated energy system and load aggregation merchants The total amount of electricity purchased constitutes the revenue source of the smart grid at the market level.
5. The multi-agent game-theoretic collaborative planning method for an airport integrated energy system under physical-information-market coupling as described in claim 4, characterized in that, The power balance P1 is represented as follows: ； In the formula: A collection of power generation equipment, including combined heat and power units and photovoltaic / wind power; For equipment At any moment The electrical power output; For the smart grid at stage k time to renewable energy operators RE Electricity purchase; For energy storage devices at time k The discharge power; For load aggregation i At stage k The electricity load demand; For energy storage devices at time k The charging power; Integrated Energy Systems for Airports (IES) In the stage time t Electricity purchased from the smart grid; The thermal balance P2 is represented as follows: ； In the formula: It is a collection of heat-generating equipment, including combined heat and power units and gas-fired boilers; For heat generation equipment j In the stage k time The heat production capacity is as follows; For load aggregation i In the stage k time Thermal power output; Integrated Energy Systems for Airports (IES) In the stage k time Heat load requirements; The constraint P3 of the combined heat and power unit is represented as follows: ； in, For cogeneration units in the stage k time Thermal power; For cogeneration units in the stage k time The electrical power; For cogeneration units in the stage The installed capacity; Thermoelectric ratio; The constraint P4 of the gas-fired boiler is represented as follows: ； in, Integrated Energy Systems for Airports (IES) China Gas Boiler in Stage k time The heating capacity; Integrated Energy Systems for Airports (IES) In the stage The installed capacity of gas-fired boilers; For gas-fired boiler efficiency; Integrated Energy Systems for Airports (IES) China Gas Boiler in Stage k time Gas consumption; The energy storage constraint P5 is expressed as: ； in, and For energy storage devices at different times +1 and time t Energy reserves; The charging efficiency at stage k; The discharge efficiency of the energy storage device in stage k; For energy storage devices at time k t The charging power; For energy storage devices at time k t The discharge power; This represents the maximum capacity of the energy storage device in stage k.
6. The multi-agent game-theoretic collaborative planning method for an airport integrated energy system under physical-information-market coupling as described in claim 5, characterized in that, The airport integrated energy system The goal is to minimize total energy costs. Its objective function is as follows: ； in, The discount rate is... Airport Integrated Energy System In the stage k The investment cost, Airport Integrated Energy System Investment in stage k, Airport Integrated Energy System The operational costs at stage k Airport Integrated Energy System Energy purchase cost at stage k Airport Integrated Energy System In the stage k Revenue from electricity sales Airport Integrated Energy System In the stage k The carbon emission costs; For the stage k The price of carbon tax; The smart grid The goal is to maximize net present value. Its objective function is as follows: in, For smart grid Electricity sales revenue, For smart grid Investment costs, For smart grid Operation and maintenance costs; The renewable energy operator RE The goal is to maximize net present value. Its objective function is as follows: in, For renewable energy operators RE Electricity sales revenue, For renewable energy operators RE Investment costs, For renewable energy operators RE Operation and maintenance costs; The load aggregator The goal is to minimize energy costs. Its objective function is as follows: 。 7. The multi-agent game-theoretic collaborative planning method for an airport integrated energy system under physical-information-market coupling as described in claim 6, characterized in that, Step S3 includes the following steps: The Stackelberg game equilibrium problem is formulated as a two-level optimization problem, as follows: The upper-level optimization problem, with the smart grid G as the leader: ； The set of equilibrium decision variables for each follower Simultaneously satisfying the following Nash equilibrium conditions: Lower-level problems: equilibrium problems in multi-follower non-cooperative games, including: For airport integrated energy system : For renewable energy operators : For load aggregator : in, The price at which electricity is sold to the smart grid. For the investment in the airport integrated energy system in stage k, The interaction power between the airport's integrated energy system and the smart grid at stage k. For renewable energy operators' investments in phase k, The actual output power of renewable energy. The interaction power between the renewable energy operator and the smart grid in stage k is represented by variables marked with an asterisk (*), which indicate the equilibrium strategies of other decision-makers. The optimization problems of the aforementioned decision-making entities must all reach their optimal state, collectively forming the Nash equilibrium solution of the lower-level game. The optimization problem involves feeding back the Nash equilibrium solution as a parameter to the upper-level leader.
8. The multi-agent game-theoretic collaborative planning method for an airport integrated energy system under physical-information-market coupling as described in claim 7, characterized in that, The construction of the multi-agent reinforcement learning algorithm in step S4 includes the following steps: Step S41: Construction of a multi-agent reinforcement learning framework; Each decision-making entity is constructed as an intelligent agent. i ,include , , , n indivual Each intelligent agent i Having a state space Action space Reward function ; Step S42: Training mechanism and algorithm selection; A centralized training and distributed execution framework is adopted. During the training process, the agent acquires global state information, while during the execution phase, it only relies on local observations. The algorithms used include one of the MADDPG, MAPPO, and QMIX algorithms. Step S43: Multi-agent training process; Initialize the policy network and evaluation network of all agents, set the hyperparameters, and create experience replay buffers for each agent to store the state-action-reward-next state tuples during the interaction process; In each training round, each agent selects an action based on the current policy, interacts with other agents in the environment and the physical-information-market coupling model, collects complete state transition samples, and stores them in their respective experience replay buffers; A centralized training method is adopted, in which a batch of samples are sampled from the experience replay buffer and the Critic and Actor networks of each agent are updated respectively; The system continuously monitors the policy change magnitude and reward function convergence of each agent. When the action distribution output by the policy network tends to stabilize and the reward function of all agents fluctuates less than a preset threshold in several consecutive rounds, the system is considered to have converged to the vicinity of Nash equilibrium, and the training process is terminated. Complete the construction of a multi-agent reinforcement learning algorithm.
9. The multi-agent game-theoretic collaborative planning method for an airport integrated energy system under physical-information-market coupling as described in claim 8, characterized in that, Each agent in step S4 i Having a state space Action space Reward function Specifically: For smart grids Building intelligent agents state space Action space reward function , For smart grid Investment capacity at stage k; For airport integrated energy system Building intelligent agents state space Action space reward function , Airport Integrated Energy System Investment capacity at stage k; For renewable energy operators Building intelligent agents state space Action space reward function , For renewable energy operators Investment capacity at stage k; for n Individual load aggregator Construct n intelligent agents state space Action space reward function .
10. The multi-agent game-theoretic collaborative planning method for an airport integrated energy system under physical-information-market coupling as described in claim 9, characterized in that, Step S5 includes the following steps: Step S51: Generate a multi-phase equipment investment timeline diagram: Based on the policy network of each agent after training convergence, the output of each decision-making agent at each planning stage is provided. The optimal decision variables were determined, resulting in a multi-period equipment investment time series diagram, including: smart grid. Optimal transmission service pricing and optimal power grid expansion investment Airport Integrated Energy System Optimal equipment investment capacity Optimal scheduling output and optimal purchase volume Renewable energy operators Optimal power plant site selection decision Optimal power plant capacity decision renewable energy operators RE Optimal price for selling electricity to the smart grid and smart grid G to renewable energy operators RE Optimal purchase volume ; Various load aggregation merchants Optimal electricity purchase from smart grid and optimal power for participation in smart grid demand response regulation ; Step S52, output the typical daily scheduling scheme: For typical daily / seasonal operating scenarios, the optimal decision variables of each decision-making entity are generated based on S51 for each hour. The scheduling schemes for multi-energy production, storage, trading, and consumption include: real-time power and heat balance schemes; charging and discharging plans for energy storage devices; energy interaction plans with the power grid; demand response participation and economic compensation settlement; Step S53, Comprehensive evaluation of the planning scheme: The generated planning scheme is evaluated from three dimensions: economic efficiency, environmental friendliness, and system reliability. Specifically, this includes: Economic assessment: Calculate the total system cost, net present value of each entity, and investment payback period indicators; Environmental assessment: Calculate the system's total carbon emissions, renewable energy penetration rate, and carbon emission reductions; Sensitivity analysis: Analyzing the impact of key parameters on the optimal decision variables; Finally, the output includes a timeline of equipment investment for multiple periods, a scheduling scheme for typical operating days, economic and environmental assessment results, and sensitivity analysis results.