Multi-industrial park IES optimization operation method containing multi-subject transactions and agents

Abstract

The invention discloses a multi-industrial park IES optimization operation method containing multi-agent transactions and agents, and the method comprises the steps: additionally arranging a part of energy production equipment and energy storage equipment in each industrial park in combination with the energy consumption characteristics of the park, and considering the power and heat load demands of each industrial park to flexibly respond to resources; forming each industrial park energy merchant to carry out energy scheduling management in each park; constructing a multi-industrial park comprehensive energy system operation framework; setting a transaction operation strategy of the multi-industrial park integrated energy system operation architecture; defining industrial park energy merchants, energy suppliers, industrial park energy operation managers, electric power market output list elements and thermal power market output list elements as corresponding reinforcement learning agents; and forming an agent multi-level transaction mode according to the set transaction operation strategy, constructing a multi-level optimization operation model of each agent, and outputting the optimal operation strategy of each agent, including the transaction price, the transaction energy quantity and the equipment operation output.

Description

Technical Field

[0001] The present invention belongs to the technical field of integrated energy systems, and particularly relates to a method for optimizing the operation of a multi-industrial park IES with multi-agent trading and intelligent agents. Background Art

[0002] Traditional industrial parks usually obtain electric energy and thermal energy from the outside to meet the energy required for the production operation process, daily office operation, etc. inside the park, and each industrial park operates independently, without energy information interaction, which will result in poor energy flexibility and high operation cost in each industrial park.

[0003] With the continuous advancement of the digital reform and low-carbon transformation of the energy system, the optimal operation of energy in smart industrial parks has received more and more attention. For example, photovoltaic power generation units, air source heat pumps, etc. can be established inside the industrial park to meet some of the electricity, heat and other load demands inside the park, which can effectively reduce the operation cost of the park. However, the energy equipment inside a single industrial park may not be able to meet the energy demand of the park. At this time, it is necessary to purchase energy from the outside. If each industrial park purchases and sells energy separately, it will lead to high transaction prices and complex transaction processes. At this time, it is necessary to establish a corresponding energy operation management company to conduct overall operation management of each industrial park under its jurisdiction and act as the representative of each industrial park to trade with the energy market. Therefore, how to conduct intelligent operation management of the production capacity, energy consumption, transactions between industrial parks, and transactions between industrial parks and the external energy market is an urgent problem to be solved at present.

[0004] Based on the above technical problems, it is necessary to design a new method for optimizing the operation of a multi-industrial park IES with multi-agent trading and intelligent agents. Summary of the Invention

[0005] In order to solve the above technical problems, the technical solution of the present invention is:

[0006] The present invention provides a method for optimizing the operation of a multi-industrial park IES with multi-agent trading and intelligent agents, including:

[0007] S1. Combine the energy consumption characteristics of each industrial park to add some production capacity equipment and energy storage equipment in each industrial park, and at the same time flexibly respond to resources considering the electricity and heat load demands of each industrial park, so as to form an energy dispatching management within each industrial park by the energy merchants in each industrial park;

[0008] S2. Establish an industrial park energy operation management company above the energy merchants in each industrial park, as the representative of the energy merchants in each industrial park, and jointly construct an operation architecture for a multi-industrial park integrated energy system with the electricity market clearing unit, energy suppliers, heat market clearing unit, and energy merchants in industrial parks;

[0009] S3. Set the trading operation strategy for the integrated energy system operation framework of multiple industrial parks, including:

[0010] When the energy merchants in each industrial park are energy producers, they output part of the electric energy and heat energy to meet the electricity and heat load demands in their respective parks. If there is an imbalance between supply and demand, then one or a combination of multiple preset flexible modes is used to achieve supply-demand balance while maximizing operational economy;

[0011] Among them, the multiple flexible modes include: the first mode is cooperation and trading among the energy merchants in each industrial park; the second mode is energy information interaction between the energy merchants in each industrial park and the industrial park energy operation management merchant, and then the industrial park energy operation management merchant conducts energy purchase and sale transactions with the electricity market clearing unit and the heat market clearing unit; the third mode is the response of flexible resources within each industrial park energy merchant.

[0012] S4. Define the energy merchants, energy suppliers, industrial park energy operation management merchants, electricity market clearing units, and heat market clearing units in each industrial park as corresponding reinforcement learning agents;

[0013] S5. Form an agent multi-level trading method according to the set trading operation strategy, and construct an optimal operation model for each agent multi-level, and output the optimal operation strategies of each agent, including trading prices, trading energy quantities, and equipment operation outputs;

[0014] Among them, the agent multi-level trading method includes: the industrial park energy operation management merchant agent, the energy supplier agent conducts transactions with the electricity market clearing unit agent and the heat market clearing unit agent; the industrial park energy operation management merchant agent conducts transactions with the energy merchant agents in each industrial park; the energy merchant agents in each industrial park conduct transactions with each other.

[0015] Furthermore, in S1, some production capacity equipment and energy storage equipment are added in each industrial park in combination with the energy consumption characteristics of the park, including:

[0016] Obtain the changing trends of the electricity load demand and heat load demand in each industrial park on a quarterly, monthly, and daily basis. When the total electricity load demand is in a high range and the time difference between the peak and trough of the electricity load demand is obvious, consider adding some power generation equipment and energy storage equipment in this industrial park, including photovoltaic power generation equipment, wind power generation equipment, and electricity storage equipment; similarly, when the total heat load demand is in a high range and the time difference between the peak and trough of the heat load demand is obvious, consider adding some heat production equipment and energy storage equipment in this industrial park, including solar thermal power generation equipment, electric heat pumps, and electricity storage equipment and heat storage equipment.

[0017] Further, in S1, considering the flexible response resources for the electricity and heat load demands of each industrial park, the energy dispatching management within each industrial park is formed by the energy merchants in each industrial park, including:

[0018] Considering the adjustability of the energy consumption time periods in the production and operation processes of each industrial park, as well as the transfer and reduction behaviors of the energy consumption for daily office and living operations in the park, analyze the demand flexible response resources, response time periods, and response capacities of each industrial park, and report them to the energy merchants in each industrial park;

[0019] As the energy dispatching management platform for each industrial park, the energy merchants in each industrial park obtain the park energy information including the operating states of the production capacity equipment and energy storage equipment, demand response information, electricity load demand, and heat load demand within each industrial park, and conduct equipment dispatching management and demand response execution within each industrial park.

[0020] Further, in S2, the operation architecture of the multi-industrial-park integrated energy system includes, from top to bottom: the first layer is the energy suppliers, the second layer is the electricity market clearing unit and the heat market clearing unit, the third layer is the industrial park energy operation management merchants, and the fourth layer is the energy merchants in each industrial park; the energy suppliers in the first layer and the industrial park energy operation management merchants in the third layer conduct energy purchase and sale transactions with the electricity market clearing unit and the heat market clearing unit in the second layer; there is also a park electricity trading platform established among the energy merchants in each industrial park for realizing electricity trading among each industrial park.

[0021] Further, in S3, when the energy merchants in each industrial park are energy producers, they output part of the electric energy and heat energy to meet the electricity and heat load demands within their respective parks. If there is an imbalance between supply and demand, then use one mode or a combination of multiple preset flexible modes to achieve the optimal operation economy while achieving the balance between supply and demand, including:

[0022] The energy merchants in each industrial park collect the historical operation parameters of the production capacity equipment and energy storage equipment within their parks, and at the same time, in combination with the external environmental conditions, establish a park energy supply prediction model; and collect the historical energy consumption parameters within their parks, and at the same time, in combination with the outdoor weather data, the changing conditions of the daily production operations in the park, the characteristics of holidays, and the characteristics of working days, establish a park load demand prediction model;

[0023] After the energy suppliers in each industrial park respectively output the electricity and heat supply values for each time period in each industrial park and the electricity and heat load demand values for each time period in each industrial park according to the energy supply prediction model and the park load demand prediction model of the park, and conduct a comparison of the supply-demand relationship. When the supply and demand are balanced, the electricity and heat loads of the park are directly met by the production capacity equipment and energy storage equipment in the energy suppliers of each industrial park. When the supply and demand are unbalanced, a variety of flexible modes are pre-set. According to the deviation range of the supply-demand imbalance, and with the goals of supply-demand balance and optimal operation economy, one of the modes or a combination of multiple modes is selected and utilized to achieve both supply-demand balance within each park and optimal operation economy.

[0024] Further, when the supply and demand are unbalanced, a variety of flexible modes are pre-set. According to the deviation range of the supply-demand imbalance, and with the goals of supply-demand balance and optimal operation economy, selecting and utilizing one of the modes or a combination of multiple modes includes:

[0025] When the supply and demand are unbalanced, if the supply-demand deviation is in the low interval range, one of the third mode or the first mode is utilized; if the supply-demand deviation is in the middle interval range, a combination of the third mode and the first mode is utilized, or the second mode is utilized; if the supply-demand deviation is in the high interval range, a combination of two of the first mode, the second mode and the third mode or a combination of all three modes is utilized;

[0026] The energy suppliers in each industrial park take the supply-demand balance and optimal operation economy of their respective parks as the goals, and select the optimal mode for trading operation in combination with the mode utilization rules when the supply and demand are unbalanced.

[0027] Further, in S4, defining the energy suppliers in each industrial park, energy suppliers, industrial park energy operation management providers, electricity market clearing units, and heat market clearing units as corresponding reinforcement learning agents includes:

[0028] Defining the energy suppliers in each industrial park as the reinforcement learning agents in each industrial park: taking the operation parameters of the production capacity equipment, electricity and heat supply values, electricity load and heat load demand values, energy storage capacity of the energy storage equipment, indoor temperature of each energy-using area in the park, production process operation parameters, electricity price, heat price, and energy interaction information of other external trading entities at time t as the state space S 1,t and taking the operation output of the production capacity equipment and energy storage equipment, energy trading volume, demand response time period, and demand response capacity in each industrial park at time t as the action space A 1,t and taking the state S at time t 1,t transferring to the state S at time t + 1 1,t+1 and taking the minimum operation cost of each industrial park as the reward function R 1,t ;

[0029] Define the energy supplier as a reinforcement learning agent: take the operating parameters of the electric and heat energy supply equipment, electricity price, heat price, weather conditions, energy price declared to the market, and declared power at time t as the state space S 2,t , and take the operating output of the electric and heat energy supply equipment and the energy trading volume at time t as the action space A 2,t , and transfer from the state S at time t 2,t to the state S at time t+1 2,t+1 , and take the maximum profit of the energy supplier as the reward function R 2,t ;

[0030] Define the energy operation and management operator of the industrial park as a reinforcement learning agent: take the electricity and heat supply values, electric load and heat load demand values of all industrial parks, the energy storage capacity of energy storage equipment, the time period and response capacity participating in demand response, the energy price declared to the market, and the declared power at time t as the state space S 3,t , and take the energy trading volume with the external market, the energy volume allocated to each industrial park energy operator, and the demand response information at time t as the action space A 3,t , and transfer from the state S at time t 3,t to the state S at time t+1 3,t+1 , and take the maximum operating income of the energy operation and management operator of the industrial park as the reward function R 3,t ;

[0031] Define the power market clearing unit and the heat market clearing unit as reinforcement learning agents: take the energy price and power declared by the energy supplier and the energy price and power declared by the energy operation and management operator of the industrial park at time t as the state space S 4,t , and take the clearing electricity price, clearing heat price, and clearing volume at time t as the action space A 4,t , and transfer from the state S at time t 4,t to the state S at time t+1 4,t+1 , and take the maximum social welfare as the reward function R 4,t .

[0032] Furthermore, in S5, the energy operation and management operator agent of the industrial park, the energy supplier agent trade with the power market clearing unit agent and the heat market clearing unit agent, including:

[0033] At the beginning of the initial quotation, the energy operation and management operator agent of the industrial park and the energy supplier agent respectively declare their energy prices and powers to the power market clearing unit agent and the heat market clearing unit agent. Subsequently, after the power market clearing unit agent and the heat market clearing unit agent process the declared data according to the quotation clearing mechanism, they output the clearing data information;

[0034] Taking the electricity market clearing unit agent and the heat market clearing unit agent as the leaders, and the industrial park energy operation management agent and the energy supplier agent as the followers, analyze the clearing data information based on the market interest equilibrium of game theory to determine whether direct clearing is possible. If the interests are not balanced, send a signal of unbalanced interests to the industrial park energy operation management agent and the energy supplier agent, and make a re-bid according to the clearing data information until the interests are balanced and the final clearing data information is output;

[0035] Transactions are carried out between the industrial park energy operation management agent and each industrial park energy agent, including:

[0036] Taking the industrial park energy operation management agent as the leader and each industrial park energy agent as the followers, the leader formulates a pricing strategy based on the transaction information declared by each follower and publishes it to each follower. Each follower adjusts its actual energy demand according to the price signal, re-declares the transaction information, and constructs and solves a principal-agent game model to output the transaction information between the two;

[0037] Transactions between each industrial park energy agent include: Defining the transactions between each industrial park energy agent as cooperative game transactions, aiming at minimizing the cost difference before and after each industrial park energy agent participates in Nash bargaining, and taking the transaction electricity price between each industrial park energy agent as the decision variable, constructing a cooperative game Nash bargaining model, and solving to output the transaction information between each industrial park energy agent.

[0038] Further, in S5, constructing a multi-layer optimal operation model for each agent, including:

[0039] According to the process of transactions between the industrial park energy operation management agent, the energy supplier agent and the electricity market clearing unit agent, the heat market clearing unit agent, determine the transaction and optimal operation objects, and combine the reward functions of the corresponding agents to form the optimal objective functions of each object, and then construct the first-layer principal-agent game optimal operation model, expressed as:

[0040]

[0041] (IEOM∪ES∪EMC∪HMC) are the participants in the game, including the industrial park energy operation management, energy suppliers, electricity market clearing units and heat market clearing units; respectively represent the transaction electricity price and heat price strategies output by the electricity market clearing unit and the heat market clearing unit to the industrial park energy operation management and energy suppliers; P t e ,P th The trading electricity volume and trading heat quantity strategies of the energy operation management provider, energy supplier, electricity market clearing unit, and heat market clearing unit in the industrial park, respectively; The operating output strategy of the i-th device in the energy supplier; I IEOM 、I ES 、I EMC 、I HMC The respective optimization objective functions of the energy operation management provider, energy supplier, electricity market clearing unit, and heat market clearing unit during trading in the industrial park, respectively;

[0042] According to the trading process between the intelligent agent of the energy operation management provider in the industrial park and the intelligent agents of each energy merchant in the industrial park, determine the trading and optimized operation objects, and form the optimization objective functions of each object by combining the reward functions of the corresponding intelligent agents, and then construct the second-layer master-slave game optimized operation model, expressed as:

[0043]

[0044] (IEOM∪IE1∪IE2∪…∪IE n ) are the participants in the game, including the energy operation management provider in the industrial park and n energy merchants in each industrial park; The trading electricity price and trading heat price strategies output by the energy operation management provider in the industrial park to the energy merchants in the industrial park, respectively; The trading electricity volume and trading heat quantity strategies between each energy merchant in the industrial park and the energy operation management provider in the industrial park, respectively; The operating output strategy of the j-th device in the energy merchant in the industrial park; P t DR The flexible resource demand response strategy in the energy merchant in the industrial park; I′ IEOM 、 The respective optimization objective functions of the energy operation management provider in the industrial park and each energy merchant in the industrial park during trading, respectively;

[0045] According to the cooperative game Nash bargaining model between the intelligent agents of each energy merchant in the industrial park, determine the trading electricity price, and form the optimization objective by combining the reward functions of the corresponding intelligent agents, and then construct the third-layer cooperative game optimized operation model, expressed as:

[0046]

[0047] C e,k,t The power generation cost of the k-th energy merchant in the industrial park at time t; C h,k,t The heat generation cost of the k-th energy merchant in the industrial park at time t; C es,k,t The energy storage cost of the k-th energy merchant in the industrial park at time t; C m,k,tIt is the electricity trading cost of the k-th industrial park energy merchant in the t period.

[0048] Furthermore, the solution methods of the first-layer master-slave game optimization operation model, the second-layer master-slave game optimization operation model, and the third-layer cooperative game optimization operation model adopt the deep reinforcement learning algorithm.

[0049] The beneficial effects of the present invention are as follows:

[0050] (1) In each industrial park, the present invention adds some production capacity equipment and energy storage equipment in combination with the energy consumption characteristics of the park, and at the same time flexibly responds to resources considering the electricity and heat load demands of each industrial park, forming the energy scheduling management within each industrial park by the energy merchants in each industrial park; an industrial park energy operation manager is established above the energy merchants in each industrial park, as the representative of the energy merchants in each industrial park, and together with the electricity market clearing unit, energy suppliers, heat market clearing unit, and industrial park energy merchants, a multi-industrial park integrated energy system operation framework is constructed; by adding production capacity equipment and energy storage equipment, the equipment can be flexibly adjusted according to the energy consumption characteristics of the park, reducing energy waste, and considering the flexible response of resources to demand, the industrial park can be flexibly adjusted according to energy prices and production operation processes; in addition, establishing an industrial park energy operation manager can achieve the overall scheduling and management of the energy merchants in each industrial park, and conduct transactions with the electricity market clearing unit and heat market clearing unit as the representative, facilitating the purchase and sale of energy with the market clearing unit when there is an imbalance between supply and demand within the industrial park energy merchants.

[0051] (2) When the energy merchants in each industrial park are energy producers, they output part of the electricity and heat to meet the electricity and heat load demands within their respective parks. If there is an imbalance between supply and demand, then one or a combination of multiple preset flexible modes is used to achieve supply-demand balance while achieving the optimal operation economy; it can achieve supply-demand balance within the industrial park and ensure the minimum operation cost of the park through multiple flexible modes in the case of imbalance within the industrial park.

[0052] (3) In the present invention, energy merchants, energy suppliers, energy operation and management merchants in industrial parks, power market clearing units, and heat market clearing units in industrial parks are defined as corresponding reinforcement learning agents; a multi-level trading method for agents is formed according to the set trading operation strategy, and a multi-level optimization operation model for each agent is constructed to output the optimal operation strategy for each agent, including trading price, trading energy quantity, and equipment operation output; it can effectively utilize the reinforcement learning agents to quickly adapt to changes in the environment and system uncertainties, such as price fluctuations, changes in supply and demand relationships, equipment operation status, etc. Each agent automatically learns the optimal decision-making strategy to ensure maintaining its respective competitive advantages in complex and changing environmental information, improving the accuracy and efficiency of decision-making, and reducing the uncertainty of human intervention; in addition, the multi-level trading method enables more effective communication and cooperation between different agents, realizes the optimal scheduling and management of resources and energy, and uses the master-slave game and cooperative game in the multi-level optimization model to optimize the operation model, and through layer-by-layer optimization of decisions, realizes the economic optimality of each agent.

[0053] Other features and advantages will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention are achieved and obtained by the structures specifically pointed out in the specification and the drawings.

[0054] To make the above objectives, features, and advantages of the present invention more obvious and understandable, the following specifically provides preferred embodiments and, in conjunction with the accompanying drawings, the detailed description is as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0056] Figure 1 It is a flowchart of a multi-industrial park IES optimization operation method of the present invention including multi-agent trading and agents;

[0057] Figure 2 It is a schematic diagram of the structure of the multi-industrial park IES of the present invention;

[0058] Figure 3 It is a schematic diagram of the principle of the first-layer master-slave game of the present invention;

[0059] Figure 4 It is a schematic diagram of the principles of the second-layer master-slave game and the third-layer cooperative game of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0061] Embodiment 1

[0062] As Figure 1 、 Figure 2 shown, Embodiment 1 of the present invention provides a method for optimizing the operation of a multi-industrial park IES with multi-agent transactions and intelligent agents, including:

[0063] S1. Combine the energy consumption characteristics of each industrial park to add some production capacity equipment and energy storage equipment, and at the same time consider the flexible response of the electricity and heat load demands of each industrial park to resources, so as to form the energy dispatch management within each industrial park by the energy merchants in each industrial park.

[0064] S2. Establish an industrial park energy operation management merchant above the energy merchants in each industrial park as the representative of the energy merchants in each industrial park, and jointly construct an operation architecture for the multi-industrial park integrated energy system with the electricity market clearing unit, energy suppliers, heat market clearing unit, and energy merchants in each industrial park.

[0065] S3. Set the trading operation strategy of the operation architecture of the multi-industrial park integrated energy system, including:

[0066] When the energy merchants in each industrial park are energy producers, they output part of the electric energy and heat energy to meet the electricity and heat load demands within their respective parks. If the supply and demand are unbalanced, then use one or a combination of multiple preset flexible modes to achieve supply-demand balance while achieving the optimal operation economy.

[0067] Among them, the multiple flexible modes include: the first mode is the cooperative transaction between the energy merchants in each industrial park; the second mode is the energy information interaction between the energy merchants in each industrial park and the industrial park energy operation management merchant, and then the industrial park energy operation management merchant conducts energy purchase and sale transactions with the electricity market clearing unit and the heat market clearing unit; the third mode is the response of flexible resources within the energy merchants in each industrial park.

[0068] S4. Define the energy merchants in each industrial park, energy suppliers, industrial park energy operation management merchants, electricity market clearing units, and heat market clearing units as corresponding reinforcement learning intelligent agents.

[0069] S5. Form an agent multi - level trading method according to the set trading operation strategy, construct an optimal operation model for each agent at multiple levels, and output the optimal operation strategies of each agent, including trading price, trading energy quantity, and equipment operation output.

[0070] Among them, the agent multi - level trading method includes: the agent of the industrial park energy operation manager trades with the agent of the energy supplier, the clearing unit agent of the electricity market, and the clearing unit agent of the heat market; the agent of the industrial park energy operation manager trades with the agents of each industrial park energy merchant; the agents of each industrial park energy merchant trade with each other.

[0071] In this embodiment, in S1, some production capacity equipment and energy storage equipment are added in each industrial park in combination with the energy consumption characteristics of the park, including:

[0072] Obtain the change trends of the electricity load demand and heat load demand in each industrial park in quarters, months, and within a day. When the total electricity load demand is in a high range and the time difference between the peak and trough of the electricity load demand is obvious, consider adding some power generation equipment and energy storage equipment in the industrial park, including photovoltaic power generation equipment, wind power generation equipment, and electricity storage equipment; similarly, when the total heat load demand is in a high range and the time difference between the peak and trough of the heat load demand is obvious, consider adding some heat production equipment and energy storage equipment in the industrial park, including solar thermal power generation equipment, electric heat pumps, and electricity storage equipment and heat storage equipment.

[0073] In actual applications, due to the differences in the scale, production process, and daily office operation methods of each industrial park, the change trends of the electricity load demand and heat load demand in each industrial park are also different. Moreover, some industrial parks have a high electricity load demand and a low heat load demand, and usually only have a heat load demand in winter. Such industrial parks are defined as power - consuming industrial parks. It is necessary to establish power generation equipment and electricity storage equipment in such parks to save the cost of purchasing electricity from outside; in addition, some industrial parks have a high heat load demand due to the operation requirements of the production process, and the electricity load demand is mainly concentrated on daily office work of personnel. Then it is necessary to establish heat production equipment and heat storage equipment near the production workshop. According to the scale and economic conditions of the park, power generation equipment and electricity storage equipment can also be established, and it is also feasible to convert electrical energy into heat energy. Therefore, each industrial park needs to add production capacity equipment and energy storage equipment in combination with the actual operation situation and economic conditions of the park.

[0074] In this embodiment, in S1, consider the flexible response resources of the electricity and heat load demands in each industrial park, and form the energy dispatch management within each industrial park by the energy merchants of each industrial park, including:

[0075] Consider the adjustability of the energy consumption period of the production operation process in each industrial park, as well as the transfer behavior and reduction behavior of the energy consumption for daily office and living operations in the park. Analyze the demand flexible response resources, response periods, and response capacities of each industrial park and report them to the energy merchants in each industrial park.

[0076] As the energy dispatching management platform for each industrial park, the energy merchants in each industrial park obtain the park energy information including the operating status of production capacity equipment and energy storage equipment, demand response information, electricity load demand, and heat load demand within each industrial park, and conduct equipment dispatching management and demand response execution within each industrial park.

[0077] It should be noted that some industrial parks have off-peak and peak production seasons. During the off-peak production season, the energy consumption is small. Appropriate energy price periods can be selected or production can be carried out when the production capacity is relatively large, thus realizing the transfer and reduction of energy consumption behavior. Or, the production operation process of some industrial parks is automatically operated. According to the production capacity situation within the park, the operating time of relevant production processes can be set, so that the energy consumption period is flexible and the operating economy is optimal.

[0078] In this embodiment, in S2, the multi-industrial park integrated energy system operation architecture includes, from top to bottom: the first layer is the energy supplier, the second layer is the electricity market clearing unit and the heat market clearing unit, the third layer is the industrial park energy operation manager, and the fourth layer is the energy merchants in each industrial park. The energy supplier in the first layer and the industrial park energy operation manager in the third layer conduct energy purchase and sale transactions with the electricity market clearing unit and the heat market clearing unit in the second layer. There is also a park electricity trading platform set up among the energy merchants in each industrial park to realize the electricity trading among each industrial park.

[0079] In this embodiment, in S3, when the energy merchants in each industrial park are energy producers, they output part of the electric energy and heat energy to meet the electricity and heat load demands within their respective parks. If the supply and demand are unbalanced, then one mode or a combination of multiple modes among the preset multiple flexible modes is used to achieve the optimal operating economy while realizing the balance between supply and demand, including:

[0080] The energy merchants in each industrial park collect the historical operation parameters of the production capacity equipment and energy storage equipment within their parks, and at the same time, combine the external environmental conditions to establish a park energy supply prediction model; and collect the historical energy consumption parameters within their parks, and at the same time, combine the outdoor weather data, the changing conditions of the daily production operations in the park, the characteristics of holidays, and the characteristics of working days to establish a park load demand prediction model.

[0081] After the energy providers in each industrial park respectively output the electricity and heat supply values and the electricity and heat load demand values for each time period in each industrial park according to the energy supply prediction model and the load demand prediction model of the park, and compare the supply-demand relationship, when the supply and demand are balanced, the power generation equipment and energy storage equipment in the energy providers of each industrial park directly meet the electricity and heat loads of the park; when the supply and demand are unbalanced, multiple flexible modes are pre-set. According to the deviation range of the supply-demand imbalance, and with the goals of supply-demand balance and optimal operation economy, one of the modes or a combination of multiple modes is selected to achieve the optimal operation economy while achieving the internal supply-demand balance in each park.

[0082] In this embodiment, when the supply and demand are unbalanced, multiple flexible modes are pre-set. According to the deviation range of the supply-demand imbalance, and with the goals of supply-demand balance and optimal operation economy, selecting and using one of the modes or a combination of multiple modes includes:

[0083] When the supply and demand are unbalanced, if the supply-demand deviation is in the low interval range, one of the third mode or the first mode is used; if the supply-demand deviation is in the middle interval range, a combination of the third mode and the first mode is used, or the second mode is used; if the supply-demand deviation is in the high interval range, a combination of two of the first mode, the second mode and the third mode or a combination of all three modes is used;

[0084] The energy providers in each industrial park aim at the supply-demand balance and optimal operation economy in their respective parks, and select the optimal mode for trading and operation in combination with the mode utilization rules when the supply and demand are unbalanced.

[0085] In actual applications, when the internal energy supply of the energy provider in the industrial park is greater than the demand, the supply-demand deviation is in the low interval. Energy storage equipment can be preferentially used for storage, or the excess energy can be used to increase the production process operation output ratio to plan for more products, or the excess energy can be sold to other energy providers in the industrial park. When the supply is less than the demand and the supply-demand deviation is in the low interval, energy storage resources can be preferentially used to make up for the missing energy, or the production process operation output ratio can be reduced to plan for fewer products, or the missing energy can be purchased from other energy providers in the industrial park; in addition, when the supply-demand deviation is in the middle interval, it indicates that one method cannot make up for the deviation, and a combination of multiple modes can be used. For example, through the combination of the first mode and the third mode, or if the mode combination brings more complexity to the production operation process, energy can be purchased or sold through the energy operation and management provider in the industrial park to the power market clearing unit and the heat market clearing unit; when the supply-demand deviation is in the high interval, multiple modes can be flexibly combined and applied to jointly achieve the supply-demand balance in the park and ensure the operation economy of the park.

[0086] In this embodiment, in step S4, the energy merchants, energy suppliers, industrial park energy operation and management merchants, electricity market clearing unit, and heat market clearing unit in each industrial park are defined as corresponding reinforcement learning agents, including:

[0087] Define the energy merchants in each industrial park as the reinforcement learning agents in each industrial park: Take the operating parameters of the production capacity equipment, electricity and heat supply values, electricity load and heat load demand values, energy storage capacity of the energy storage equipment, indoor temperature of each energy - using area in the park, production process operating parameters, electricity price, heat price, and energy interaction information of other external trading entities at time t as the state space S 1,t Take the operating output of the production capacity equipment and energy storage equipment, energy trading volume, demand response period, and demand response capacity in each industrial park at time t as the action space A 1,t Take the state S at time t 1,t Transfer to the state S at time t + 1 1,t+1 And take the minimum operating cost of each industrial park as the reward function R 1,t ;

[0088] Define the energy supplier as a reinforcement learning agent: Take the operating parameters of the electric energy and heat energy supply equipment, electricity price, heat price, weather conditions, energy prices declared to the market, and declared power at time t as the state space S 2,t Take the operating output of the electric energy and heat energy supply equipment, energy trading volume at time t as the action space A 2,t Take the state S at time t 2,t Transfer to the state S at time t + 1 2,t+1 And take the maximum profit of the energy supplier as the reward function R 2,t ;

[0089] Define the industrial park energy operation and management merchant as a reinforcement learning agent: Take the electricity and heat supply values, electricity load and heat load demand values, energy storage capacity of the energy storage equipment, periods and response capacities participating in demand response, energy prices declared to the market, and declared power of all industrial parks at time t as the state space S 3,t Take the energy trading volume with the external market, the energy quantity allocated to the energy merchants in each industrial park, and the demand response information at time t as the action space A 3,t Take the state S at time t 3,t Transfer to the state S at time t + 1 3,t+1 And take the maximum operating revenue of the industrial park energy operation and management merchant as the reward function R 3,t ;

[0090] Define the power market clearing unit and the heat market clearing unit as reinforcement learning agents: regard the energy prices and powers declared by energy suppliers at time t, and the energy prices and powers declared by industrial park energy operation and management providers as the state space S 4,t Regard the clearing electricity price, clearing heat price, and clearing volume at time t as the action space A 4,t From the state S at time t 4,t Transfer to the state S at time t+1 4,t+1 And use the maximum social welfare as the reward function R 4,t .

[0091] It should be noted that reinforcement learning tasks are generally described using Markov decision processes, corresponding to a quadruple of state space S, action space A, state transition matrix P, and reward function R. The agent Agent is in the environment and calls the environment it perceives the state; the actions that the agent Agent can take constitute the action space; when a certain action acts on the current state, the environment will transfer from the current state to another state according to the state transition matrix, and at the same time the environment will feedback the corresponding reward to the agent Agent according to the reward function. The agent Agent continuously interacts with the environment to obtain the corresponding action strategy, and the quality of the action strategy depends on the cumulative reward obtained after long-term execution of this strategy.

[0092] The energy operation management within each industrial park energy provider agent can be regarded as an optimization problem that changes with the environment. The goal of this agent is to determine the action decisions to be executed according to its own goals by interacting with the environment information, and then minimize the operation cost of the industrial park by optimizing the internal production capacity equipment, energy storage equipment, and flexible resources. Each industrial park energy provider needs to interact with other industrial park energy providers, and also needs to interact with the power market clearing unit and the heat market clearing unit. At the same time, there are external energy suppliers interacting with the power market clearing unit and the heat market clearing unit. Therefore, it is also necessary to establish the corresponding agents for energy suppliers, industrial park energy operation and management providers, power market clearing units, and heat market clearing units. The environments, states, and reward functions of each agent are different, but information interaction is required between each agent. Therefore, when each agent performs optimal operation, the state space also involves the relevant information of other agents, jointly forming the operation architecture of a multi-agent multi-industrial park integrated energy system.

[0093] As Figure 3 , Figure 4 shown, in this embodiment, in S5, the industrial park energy operation and management provider agent, the energy supplier agent trade with the power market clearing unit agent and the heat market clearing unit agent, including:

[0094] At the beginning of the initial quotation, the intelligent agent of the industrial park energy operation management and the intelligent agent of the energy supplier respectively declare their respective energy prices and powers to the clearing unit intelligent agent of the electricity market and the clearing unit intelligent agent of the heat market. Subsequently, after the clearing unit intelligent agent of the electricity market and the clearing unit intelligent agent of the heat market process the declared data according to the quotation clearing mechanism, the cleared data information is output;

[0095] Regarding the clearing unit intelligent agent of the electricity market and the clearing unit intelligent agent of the heat market as leaders, and the intelligent agent of the industrial park energy operation management and the intelligent agent of the energy supplier as followers, analyze the cleared data information based on the market interest equilibrium of game theory to determine whether direct clearing is possible. If the interests are not balanced, send a signal of unbalanced interests to the intelligent agent of the industrial park energy operation management and the intelligent agent of the energy supplier, and conduct re-quotation based on the cleared data information until the interests are balanced and the final cleared data information is output;

[0096] Transactions are conducted between the intelligent agent of the industrial park energy operation management and each intelligent agent of the industrial park energy merchant, including:

[0097] Regarding the intelligent agent of the industrial park energy operation management as the leader and each intelligent agent of the industrial park energy merchant as the follower, the leader formulates a pricing strategy based on the transaction information declared by each follower and distributes it to each follower. Each follower adjusts its actual energy demand according to the price signal, re-declares the transaction information, and constructs and solves a principal-agent game model to output the transaction information between the two;

[0098] Transactions between each intelligent agent of the industrial park energy merchant include: defining the transactions between each intelligent agent of the industrial park energy merchant as cooperative game transactions, aiming at minimizing the cost difference before and after each intelligent agent of the industrial park energy merchant participates in Nash bargaining, using the transaction electricity price between each intelligent agent of the industrial park energy merchant as the decision variable, constructing a cooperative game Nash bargaining model, and solving to output the transaction information between each intelligent agent of the industrial park energy merchant.

[0099] It should be noted that the Nash bargaining theory belongs to the scope of cooperative games and can help decision-makers achieve fair Pareto optimal interest distribution. In response to problems with interest conflicts among participants with interactive characteristics, Nash bargaining can balance the interests of the whole and the individual.

[0100] The cooperative game Nash bargaining model is expressed as:

[0101]

[0102] is the cost when the industrial park energy merchant k does not participate in the bargaining when each industrial park energy merchant forms a cooperative alliance; The cost for the industrial park energy merchant k to participate in the bargaining when the energy merchants in each industrial park form a cooperation alliance; The optimal cost when the energy merchants in each industrial park form a cooperation alliance; The electricity sales revenue of the industrial park energy merchant k to other industrial park energy merchants when the energy merchants in each industrial park form a cooperation alliance; The transaction electricity price among the industrial park energy merchants when the energy merchants in each industrial park form a cooperation alliance; The optimal transaction electricity quantity among the industrial park energy merchants when the energy merchants in each industrial park form a cooperation alliance.

[0103] In this embodiment, in the step S5, constructing a multi - layer optimal operation model for each agent includes:

[0104] According to the transaction process between the industrial park energy operation management agent, the energy supplier agent and the electricity market clearing unit agent, the heat market clearing unit agent, determining the transaction and optimal operation objects, and combining the reward functions of the corresponding agents to form the optimal objective functions of each object, and then constructing the first - layer master - slave game optimal operation model, expressed as:

[0105]

[0106] (IEOM∪ES∪EMC∪HMC) are the participants in the game, including the industrial park energy operation management merchant, the energy supplier, the electricity market clearing unit and the heat market clearing unit; respectively represent the transaction electricity price and heat price strategies output by the electricity market clearing unit and the heat market clearing unit to the industrial park energy operation management merchant and the energy supplier; P t e ,P t h respectively are the transaction electricity quantity and transaction heat quantity strategies of the industrial park energy operation management merchant, the energy supplier and the electricity market clearing unit, the heat market clearing unit; is the operation output strategy of the i - th device in the energy supplier; I IEOM 、I ES 、I EMC 、I HMC respectively are the respective optimal objective functions when the industrial park energy operation management merchant, the energy supplier trade with the electricity market clearing unit and the heat market clearing unit;

[0107] According to the transaction process between the industrial park energy operation management agent and each industrial park energy merchant agent, determining the transaction and optimal operation objects, and combining the reward functions of the corresponding agents to form the optimal objective functions of each object, and then constructing the second - layer master - slave game optimal operation model, expressed as:

[0108]

[0109] (IEOM ∪ IE1 ∪ IE2 ∪ … ∪ IE n ) are the participants in the game, including the industrial park energy operation and management operator and n industrial park energy merchants; are the trading electricity price and trading heat price strategies output by the industrial park energy operation and management operator to the industrial park energy merchants respectively; are the trading electricity volume and trading heat volume strategies between each industrial park energy merchant and the industrial park energy operation and management operator respectively; is the operation output strategy of the j-th device in the industrial park energy merchant; P t DR is the flexible resource demand response strategy in the industrial park energy merchant; I I ′ EOM 、 are the respective optimization objective functions of the industrial park energy operation and management operator and each industrial park energy merchant when trading;

[0110] According to the cooperative game Nash bargaining model among the industrial park energy merchant agents, determine the trading electricity price, and form an optimization objective by combining the reward functions of the corresponding agents, and then construct the third-layer cooperative game optimization operation model, expressed as:

[0111]

[0112] C e,k,t is the power generation cost of the k-th industrial park energy merchant at time t; C h,k,t The heat generation cost of the k-th industrial park energy merchant at time t; C es,k,t is the energy storage cost of the k-th industrial park energy merchant at time t; C m,k,t is the electricity trading cost of the k-th industrial park energy merchant at time t.

[0113] In this embodiment, the solution methods of the first-layer master-slave game optimization operation model, the second-layer master-slave game optimization operation model, and the third-layer cooperative game optimization operation model adopt the deep reinforcement learning algorithm.

[0114] In several embodiments provided by the present application, it should be understood that the disclosed systems and methods can also be implemented in other ways. The system embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of systems, methods, and computer program products according to multiple embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0115] In addition, each functional module in various embodiments of the present invention can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part. If the function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories, random access memories, magnetic disks, or optical discs that can store program codes.

[0116] Taking the above ideal embodiments of the present invention as an inspiration, through the above description, relevant staff can completely make various changes and modifications without departing from the technical idea of this invention. The technical scope of this invention is not limited to the content in the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A multi-industrial park IES optimal operation method containing multi-agent transactions and agents, characterized in that, Including: S1. Combine the energy consumption characteristics of each industrial park to add some production capacity equipment and energy storage equipment, and at the same time flexibly respond to resources considering the electricity and heat load demands of each industrial park, so as to form the energy dispatching management within each industrial park by the energy merchants in each industrial park; S2. Establish an energy operation management merchant for industrial parks above the energy merchants in each industrial park, which serves as the representative of the energy merchants in each industrial park, and jointly construct an operation framework for the integrated energy system of multiple industrial parks with the electricity market clearing unit, energy suppliers, heat market clearing unit, and energy merchants in industrial parks; S3. Set the trading operation strategies for the operation framework of the integrated energy system of multiple industrial parks, including: When the energy merchants in each industrial park are energy producers, they output part of the electric energy and heat energy to meet the electricity and heat load demands within their respective parks. If the supply and demand are unbalanced, then use one or a combination of multiple preset flexible modes to achieve supply-demand balance while achieving the optimal operation economy; Among them, the multiple flexible modes include: the first mode is the cooperative trading among the energy merchants in each industrial park; the second mode is the energy information interaction between the energy merchants in each industrial park and the energy operation management merchant for industrial parks, and then the energy operation management merchant for industrial parks conducts energy purchase and sale transactions with the electricity market clearing unit and the heat market clearing unit; the third mode is the response of flexible resources within each energy merchant in the industrial park; S4. Define the energy merchants in each industrial park, energy suppliers, energy operation management merchants for industrial parks, electricity market clearing units, and heat market clearing units as corresponding reinforcement learning agents; S5. Form an agent multi-level trading method according to the set trading operation strategies, and construct an optimal operation model for each agent multi-level, and output the optimal operation strategies of each agent, including trading prices, trading energy quantities, and equipment operation outputs; Among them, the agent multi-level trading method includes: the energy operation management merchant agent for industrial parks, the energy supplier agent conducts transactions with the electricity market clearing unit agent and the heat market clearing unit agent; the energy operation management merchant agent for industrial parks conducts transactions with the energy merchant agents in each industrial park; the energy merchant agents in each industrial park conduct transactions with each other.

2. The method for optimizing the operation of IES in multiple industrial parks according to claim 1, wherein In the above S1, adding some production capacity equipment and energy storage equipment in each industrial park in combination with the energy consumption characteristics of the park includes: Obtain the changing trends of the electricity load demand and heat load demand in each industrial park on a quarterly, monthly, and daily basis. When the total electricity load demand is in a high range and the time difference between the peak and trough of the electricity load demand is obvious, consider adding some power generation equipment and energy storage equipment within the industrial park, including photovoltaic power generation equipment, wind power generation equipment, and electricity storage equipment; similarly, when the total heat load demand is in a high range and the time difference between the peak and trough of the heat load demand is obvious, consider adding some heat production equipment and energy storage equipment within the industrial park, including solar thermal power generation equipment, electric heat pumps, and electricity storage equipment, heat storage equipment.

3. The multi-industrial park IES optimal operation method according to claim 1, characterized in that, In S1, considering the flexible response resources for the electricity and heat load demands of each industrial park, the energy merchants in each industrial park conduct energy dispatching management within their respective parks, including: Considering the adjustability of the energy consumption periods in the production and operation processes of each industrial park, as well as the transfer and reduction behaviors of the energy consumption for daily office and living operations in the park, analyzing the flexible response resources, response periods, and response capacities of each industrial park, and reporting them to the energy merchants in each industrial park; As the energy dispatching management platform for each industrial park, the energy merchants in each industrial park obtain the park energy information including the operating states of the production capacity equipment and energy storage equipment, demand response information, electricity load demand, and heat load demand within each industrial park, and conduct equipment dispatching management and demand response execution within each industrial park.

4. The multi-industrial park IES optimization operation method according to claim 1, characterized in that In S2, the operation architecture of the multi-industrial-park integrated energy system includes, from top to bottom: the first layer is the energy supplier, the second layer is the electricity market clearing unit and the heat market clearing unit, the third layer is the industrial park energy operation manager, and the fourth layer is the energy merchants in each industrial park; the energy supplier in the first layer and the industrial park energy operation manager in the third layer conduct energy purchase and sale transactions with the electricity market clearing unit and the heat market clearing unit in the second layer; there is also a park electricity trading platform set up among the energy merchants in each industrial park to realize electricity trading among the industrial parks.

5. The method for optimizing the operation of IES in multiple industrial parks according to claim 1, characterized in that, In S3, when the energy merchants in each industrial park are energy producers, they output part of the electric energy and heat energy to meet the electricity and heat load demands within their respective parks. If the supply and demand are unbalanced, then one or a combination of multiple preset flexible modes is used to achieve the optimal operation economy while realizing the balance between supply and demand, including: The energy merchants in each industrial park collect the historical operation parameters of the production capacity equipment and energy storage equipment within their parks, and at the same time, combined with the external environmental conditions, establish a park energy supply prediction model; and collect the historical energy consumption parameters within their parks, and at the same time, combined with the outdoor weather data, the changing conditions of the daily production and operation in the park, and the characteristics of holidays and working days, establish a park load demand prediction model; After the energy merchants in each industrial park respectively output the electricity and heat supply values and the electricity and heat load demand values for each period in each industrial park according to the park energy supply prediction model and the park load demand prediction model, and conduct a comparison of the supply and demand relationship. When the supply and demand are balanced, the electricity and heat loads in the park are directly met by the production capacity equipment and energy storage equipment within the energy merchants in each industrial park; when the supply and demand are unbalanced, then multiple flexible modes are preset, and according to the deviation range of the supply and demand imbalance, and with the goals of supply and demand balance and optimal operation economy, one or a combination of multiple modes is selected to achieve the optimal operation economy while realizing the balance between supply and demand within each park.

6. The method for optimizing the operation of the multi-industrial park IES according to claims 1 and 5, characterized in that, When the supply and demand are unbalanced, then multiple flexible modes are preset, and according to the deviation range of the supply and demand imbalance, and with the goals of supply and demand balance and optimal operation economy, one or a combination of multiple modes is selected, including: When the supply-demand is unbalanced, if the supply-demand deviation is in the low range, one of the third mode or the first mode is utilized; if the supply-demand deviation is in the medium range, the combination of the third mode and the first mode is utilized, or the second mode is utilized; if the supply-demand deviation is in the high range, the combination of any two of the first mode, the second mode and the third mode or the combination of all three modes is utilized. Energy merchants in each industrial park aim at the optimal supply-demand balance and operation economy of their respective parks, and select the optimal mode for trading operation in combination with the mode utilization rules when the supply-demand is unbalanced.

7. The method for optimizing the operation of IES in multiple industrial parks according to claim 1, wherein, In step S4, each energy merchant in the industrial park, the energy supplier, the energy operation management entity in the industrial park, the power market clearing unit, and the heat market clearing unit are defined as corresponding reinforcement learning agents, including: Define the energy merchants in each industrial park as the reinforcement learning agents in each industrial park: take the operating parameters of the production capacity equipment, electricity and heat supply values, electricity load and heat load demand values, energy storage capacity of the energy storage equipment, indoor temperature of each energy-consuming area in the park, production process operating parameters, electricity price, heat price, and energy interaction information of other external trading entities within each industrial park at time t as the state space S 1,t , take the operating output of the production capacity equipment and energy storage equipment, energy trading volume, demand response period, and demand response capacity within each industrial park at time t as the action space A 1,t , take the state S at time t 1,t transfer to the state S at time t+1 1,t+1 , and take the minimum operating cost of each industrial park as the reward function R 1,t ; Define the energy supplier as a reinforcement learning agent: Take the operating parameters of the power and heat supply equipment, electricity price, heat price, weather conditions, energy price declared to the market, and declared power at time t as the state space S 2,t , and take the operating output of the power and heat supply equipment and the energy trading volume at time t as the action space A 2,t , and transfer from the state S at time t 2,t to the state S at time t+1 2,t+1 , and take the maximum profit of the energy supplier as the reward function R 2,t ; Define the energy operation manager of the industrial park as a reinforcement learning agent: regard the electricity and heat supply values, electricity and heat load demand values, energy storage capacity of energy storage devices, periods and response capacities participating in demand response, energy prices and declared powers declared to the market of all industrial parks at time t as the state space S 3,t , regard the energy trading volume with the external market, the energy volume allocated to each industrial park energy merchant, and the demand response information at time t as the action space A 3,t , regard the state S at time t 3,t transfer to the state S at time t + 1 3,t+1 , and take the maximum operating income of the energy operation manager of the industrial park as the reward function R 3,t ; Define the electricity market clearing unit and the heat market clearing unit as reinforcement learning agents: regard the energy prices and powers declared by energy suppliers at time t and the energy prices and powers declared by industrial park energy operation and management operators as the state space S 4,t , regard the clearing electricity price, the clearing heat price and the clearing volume at time t as the action space A 4,t , regard the state S at time t 4,t transfer to the state S at time t+1 4,t+1 , and take the maximum social welfare as the reward function R 4,t .

8. The method for optimizing the operation of IES in multiple industrial parks according to claim 1, wherein In step S5, the energy operation management entity agent in the industrial park, the energy supplier agent conduct transactions with the power market clearing unit agent and the heat market clearing unit agent, including: At the beginning of the initial quotation, the energy operation management entity agent in the industrial park and the energy supplier agent respectively declare their respective energy prices and powers to the power market clearing unit agent and the heat market clearing unit agent. Subsequently, after the power market clearing unit agent and the heat market clearing unit agent process the declared data according to the quotation clearing mechanism, the cleared data information is output. Regarding the power market clearing unit agent and the heat market clearing unit agent as leaders, and the energy operation management entity agent in the industrial park and the energy supplier agent as followers, analyze the cleared data information based on the market interest balance of game theory to determine whether direct clearing is possible. If the interests are unbalanced, send a signal of unbalanced interests to the energy operation management entity agent in the industrial park and the energy supplier agent, and conduct re-quotation according to the cleared data information until the interests are balanced and the final cleared data information is output. The transaction between the energy operation management entity agent in the industrial park and each energy merchant agent in the industrial park includes: Regarding the energy operation management entity agent in the industrial park as the leader and each energy merchant agent in the industrial park as the followers, the leader formulates a pricing strategy based on the transaction information declared by each follower and issues it to each follower. Each follower adjusts its actual energy demand according to the price signal, re-declares the transaction information, and constructs and solves a master-slave game model to output the transaction information between the two. The transaction between each energy merchant agent in the industrial park includes: defining the transaction between each energy merchant agent in the industrial park as a cooperative game transaction, aiming at minimizing the cost difference before and after each energy merchant agent in the industrial park participates in the Nash bargaining, taking the transaction electricity price between each energy merchant agent in the industrial park as the decision variable, constructing a cooperative game Nash bargaining model, and solving to output the transaction information between each energy merchant agent in the industrial park.

9. The multi-industrial park IES optimization operation method according to claim 1, characterized in that In step S5, construct a multi-layer optimal operation model for each agent, including: Based on the trading process among the intelligent agent of the industrial park energy operation management provider, the intelligent agent of the energy supplier, the clearing unit intelligent agent of the electricity market, and the clearing unit intelligent agent of the heat market, determine the trading and optimal operation objects, and form the optimal objective functions of each object by combining the reward functions of the corresponding intelligent agents, and then construct the first-layer master-slave game optimal operation model, which is expressed as: (IEOM ∪ ES ∪ EMC ∪ HMC) are the participants in the game, including the industrial park energy operation and management provider, energy supplier, electricity market clearing unit, and heat market clearing unit; respectively represent the trading electricity price and heat price strategies output by the electricity market clearing unit and heat market clearing unit to the industrial park energy operation and management provider and energy supplier; P t e , P t h are respectively the trading electricity quantity and trading heat quantity strategies between the industrial park energy operation and management provider, energy supplier and the electricity market clearing unit and heat market clearing unit; is the operation output strategy of the i-th device in the energy supplier; I IEOM , I ES , I EMC , I HMC are respectively the respective optimization objective functions when the industrial park energy operation and management provider, energy supplier trade with the electricity market clearing unit and heat market clearing unit; Based on the trading process between the intelligent agent of the industrial park energy operation management provider and each intelligent agent of the industrial park energy supplier, determine the trading and optimal operation objects, and form the optimal objective functions of each object by combining the reward functions of the corresponding intelligent agents, and then construct the second-layer master-slave game optimal operation model, which is expressed as: (IEOM ∪ IE1 ∪ IE2 ∪ … ∪ IE n ) are the participants in the game, including the industrial park energy operation and management provider and n industrial park energy providers; are respectively the trading electricity price and trading heat price strategies output by the industrial park energy operation and management provider to the industrial park energy providers; are respectively the trading electricity volume and trading heat volume strategies between each industrial park energy provider and the industrial park energy operation and management provider; is the operation output strategy of the j-th device in the industrial park energy provider; P t DR is the flexible resource demand response strategy in the industrial park energy provider; I I ′ EOM 、 are respectively the respective optimization objective functions when the industrial park energy operation and management provider trades with each industrial park energy provider; Based on the cooperative game Nash bargaining model among the intelligent agents of each industrial park energy supplier, determine the trading electricity price, and form the optimization objective by combining the reward functions of the corresponding intelligent agents, and then construct the third-layer cooperative game optimal operation model, which is expressed as: C e,k,t is the electricity generation cost of the k-th industrial park energy provider at time t; C h,k,t is the heat generation cost of the k-th industrial park energy provider at time t; C es,k,t is the energy storage cost of the k-th industrial park energy provider at time t; C m,k,t is the electricity trading cost of the k-th industrial park energy provider at time t.

10. The multi-industrial park IES optimization operation method according to claim 9, characterized in that The solution methods of the first-layer master-slave game optimal operation model, the second-layer master-slave game optimal operation model, and the third-layer cooperative game optimal operation model adopt the deep reinforcement learning algorithm.

Images