Interconnected micro-grid energy management and transaction method under user demand response

Abstract

The invention belongs to the technical field of energy management, and provides an integrated energy micro-grid energy management and transaction method under user demand response, which comprises the following steps of: 1, establishing an integrated energy micro-grid energy equipment model, a schedulable model of flexible loads such as an electric vehicle and a variable frequency air conditioner, and an operation optimization model of the schedulable model participating in market transaction; step 2, constructing a neural network required by MATD3; step 3, according to the model established in the step 1, utilizing an energy management center to interact with an interconnected micro-grid simulation environment, putting historical information obtained by interaction into an experience pool, and utilizing TD3 to realize optimization of a strategy network and an evaluation network; and step 4, controlling the micro-grid according to the optimized strategy network obtained in the step 3 so that the micro-grid can participate in the end-to-end market transaction. According to the method, nearby consumption of distributed energy is facilitated, redundant capacity and renewable energy of the comprehensive energy micro-grid are further utilized, and the economical efficiency of a micro-grid optimization scheduling strategy is taken into account.

Description

Technical Field

[0001] The present invention belongs to the field of energy management and trading technology, and more specifically, relates to an interconnected microgrid energy management and trading method under user demand response. Background Art

[0002] In order to cope with the challenges brought by energy crisis and environmental pollution, distributed energy resources (DER) are gradually being integrated into the distribution system. Multi-energy microgrid (MEMG) has gradually become one of the main structures for improving the reliability of power systems and effectively utilizing DER due to its advantages of high efficiency, low pollution and flexible configuration. However, due to the characteristics of localized power supply of microgrids, the operating capacity of a single microgrid is very limited, and its anti-interference ability is easily affected by changes in the operating environment. Therefore, by coordinating the operation of multiple microgrids and realizing peer-to-peer (P2P) energy transactions between microgrids, a local energy market of small-scale interconnected microgrids is formed, which can significantly improve the energy utilization of the system. In the P2P trading mode, the transaction price is set by the P2P trading platform or determined by negotiation between the two parties, without the intervention of the distribution system operator (DSO). This model is expected to bring significant economic benefits to energy producers, while achieving a more balanced local energy supply and demand and improving the reliability of the power system.

[0003] At present, the solution to the optimization problem of interconnected microgrids mainly relies on traditional model-based optimization methods such as Mixed-Integer Linear Programming (MILP) and Alternating Direction Method of Multipliers (ADMM). However, considering the coupled interaction of multiple energy sources such as electricity, thermal energy and natural gas energy, as well as the demand response mechanism of flexible loads, it is extremely challenging to build an accurate and efficient physical model. The power flow and energy coupling relationship are affected by many complex factors, and the model-based method has significant limitations in scalability.

[0004] In recent years, deep reinforcement learning (DRL) has attracted great attention in academia. DRL combines deep neural network (DNN) with reinforcement learning (RL) technology and is a model-free optimization method. The agent continuously learns and optimizes decision-making strategies by continuously interacting with the environment and leveraging accumulated experience to achieve optimal scheduling or behavior performance. Existing studies have successfully applied various single-agent RL methods to a wide range of application scenarios in smart grids. However, the core feature of the end-to-end trading model is the coordinated energy management between multiple interacting agents, which highlights the importance and necessity of studying multi-agent methods. Summary of the invention

[0005] In view of the problems existing in the existing technology of microgrid energy management and trading, the present invention proposes an end-to-end trading method for interconnected microgrids under demand response, which can achieve safe, stable and economical operation of microgrids under the influence of various random factors when participating in the local market.

[0006] To achieve the above object, the present invention adopts the following technical solution:

[0007] A method for energy management and trading of interconnected microgrids under user demand response includes three integrated energy microgrid systems with different load types, wherein the integrated energy microgrid system includes an energy management center, energy conversion equipment, energy storage equipment and flexible loads, wherein the energy conversion equipment includes a gas turbine, a waste heat boiler, a gas boiler, an electric heat pump, and a new energy power generation device, wherein the energy storage equipment includes a battery and a heat storage tank, wherein the flexible load includes an electric vehicle load aggregator and an air conditioning load aggregator, wherein the energy management center is an energy and information integration center, and externally undertakes distribution network operators, gas suppliers and end-to-end trading platforms, and can obtain information such as electricity price, natural gas price and trading plan issued by the trading platform, and control the purchase and trading volume of electric energy and natural gas energy, and internally can issue response invitations to flexible loads to adjust the inherent energy consumption behavior of users and control the production plan of energy equipment in the system, wherein the energy trading and management method specifically includes the following steps:

[0008] Step 1: In the end-to-end transaction mode, the microgrid energy management center usually aims to minimize its own power supply cost and maximize transaction revenue, and establishes an operation optimization model for the integrated energy microgrid to participate in local end-to-end market transactions:

[0009] Step 1.1, establish a scheduling model for energy production equipment and energy storage equipment in a comprehensive energy microgrid;

[0010] Step 1.2, establish the load response characteristic model of electric vehicles and variable frequency air conditioners in the integrated energy microgrid;

[0011] Step 1.3: Establish an end-to-end local market transaction mechanism based on auction theory

[0012] Step 1.4: Establish a dispatch optimization model and process for integrated energy microgrids to participate in the local end-to-end market;

[0013] Step 2: Build the deep neural network required for the Multi-Agent TwinDelayed Deep Deterministic Policy Gradient Algorithm (MATD3):

[0014] MATD3 is a classic multi-agent reinforcement learning algorithm that adopts the method of "centralized training and distributed execution". It is applicable to unstable environments and can be used in cooperative, competitive or mixed cooperative and competitive multi-agent environments. Each agent in MATD3 consists of a current policy network, a target policy network, two comment networks 1 and 2, and two target evaluation networks 1 and 2. The six neural networks are all fully connected neural networks, including input layers, hidden layers, and output layers. The current policy network and the target policy network have the same structure, and the evaluation network 1, evaluation network 2, target evaluation network 1 and target evaluation network 2 have the same structure. In addition, each layer of the network uses the nonlinear mapping ability of deep dense neural networks to incorporate the original input x into each hidden layer of the network, thereby retaining the information of the input x;

[0015] Step 3: Based on the model established in step 1, the energy management center is used to interact with the interconnected integrated energy microgrid simulation environment to obtain historical information and store it in the experience pool, and MATD3 is used to implement iterative updates and optimizations of the strategy network, evaluation network 1, and evaluation network 2;

[0016] Since the initial value of the model weight is randomly set at the beginning of training, if a larger learning rate is used, the model may fluctuate during the optimization process, affecting its convergence stability. Therefore, the cosine annealing method is used to adjust the learning rate;

[0017] Step 4: Implement energy management and energy trading solutions for the integrated energy microgrid based on the strategy network obtained in step 3:

[0018] At any decision-making moment in the operation of the system, the decision-making moment, the operating status of the integrated energy microgrid, the electricity trading volume and electricity trading quotation and other information are normalized and input into the strategy network of MATD3. After forward propagation, the action information output by the strategy network is denormalized to obtain the actions that the system can take at the current decision moment. The operating cost, transaction income and incentive cost obtained by the integrated energy microgrid after repeatedly performing the above operations in a scheduling cycle are observed.

[0019] The technical solution is further optimized, and the step 1.1 described in establishing the scheduling model of energy production equipment and energy storage equipment of the integrated energy microgrid is specifically performed as follows:

[0020] A thermal power network is established within each microgrid, in which the thermal network includes gas boilers, waste heat boilers, electric heat pumps, thermal energy storage and thermal loads, and the power network includes micro gas turbines, renewable energy, electric energy storage, flexible loads (electric vehicles, air conditioners) and rigid electric loads.

[0021] Gas turbine operating constraints:

[0022]

[0023] in, and are the electric power and thermal power generated by the gas turbine in the i-th microgrid during the k-th period, respectively; The natural gas power consumed by the gas turbine; is the calorific value of natural gas; and is the power generation efficiency and heat generation efficiency of the gas turbine; P i gt,min and P i gt,max The upper and lower limits of the gas turbine output electric power;

[0024] Gas boiler operation constraints:

[0025]

[0026] in, The thermal power generated by the gas boiler; The natural gas power consumed by the gas boiler; is the calorific value of natural gas; is the heat generation efficiency of the gas boiler; and The upper and lower limits of the heating power of the gas boiler;

[0027] Waste heat boiler operation constraints:

[0028]

[0029]

[0030] in, is the thermal power generated by the waste heat boiler in the i-th microgrid during the k-time period; The thermal power absorbed by the waste heat boiler; is the heat generation efficiency of the gas boiler; and The upper and lower limits of the heat generation capacity of the waste heat boiler;

[0031] Electric heat pump operation constraints:

[0032]

[0033] in, is the heating power of the electric heat pump; P i eb The electrical power consumed by the electric heat pump; is the heating efficiency of the electric heat pump; and The upper and lower limits of the heat generation power of the electric heat pump;

[0034] Thermal storage tank operation constraints:

[0035]

[0036] in, They represent the power of the heat storage tank, respectively. A positive value indicates heat release, a negative value indicates heat storage, and a value of 0 indicates idleness. is the rated capacity of the heat storage tank; and Indicates the maximum heat storage and release power of the heat storage tank; is the heat storage state of the heat storage tank; Indicates the heat storage and release coefficient of the heat storage tank; and The upper and lower limits of the heat storage tank state;

[0037] Battery operation constraints:

[0038]

[0039]

[0040] in, is the power of the battery, a positive value indicates discharge, a negative value indicates charging, and 0 indicates idle; is the rated capacity of the electric energy storage; P i ch,max and P i ds,max Indicates the maximum charging and discharging power of the heat storage tank; is the state of charge of the electrical energy storage; Indicates the charge and discharge coefficient of the battery; and The upper and lower limits of the battery status.

[0041] The technical solution is further optimized, and the response characteristic model of the electric vehicle aggregator and the variable frequency air conditioner aggregator within the integrated energy microgrid in step 1.2 is specifically as follows:

[0042] The electric vehicle aggregator is the aggregation of all charging piles in a county microgrid, responsible for managing the charging and discharging scheduling of the electric vehicle cluster in the microgrid; the electric vehicle aggregator is composed of independent electric vehicles, and the power of the electric vehicles participating in the scheduling satisfies the following formula:

[0043]

[0044] in, and are the electric quantity and electric power of the electric vehicle respectively; and are the maximum and minimum power values ​​of the nth vehicle respectively; and They represent the minimum and maximum power of electric vehicles based on the charging and discharging efficiency of the charging pile; η ev The charging and discharging efficiency of electric vehicles; comprehensively consider the information of charging piles, car owners and EVs, and establish the upper and lower limits of the power when a single electric vehicle participates in scheduling;

[0045] The Minkowski addition method is used to obtain the upper and lower boundaries of the EV cluster dispatchability capacity for the electric vehicle charging and discharging power boundary and the battery safety power boundary to reduce the dimension of the decision variables.

[0046]

[0047] Among them, P i eva,max / min and They are the upper and lower limits of power and the upper and lower limits of electricity for electric vehicle aggregators; is a {0,1} variable, indicating the off-grid status of the electric vehicle;

[0048] The power of the electric vehicle aggregator includes the power of all grid-connected electric vehicles, but the off-grid behavior of electric vehicles will affect the power of the cluster. In order to solve the instantaneous step power phenomenon, the dynamic power of the EV cluster needs to be corrected;

[0049]

[0050]

[0051] in, To correct the charge, The expected off-grid power consumption of electric vehicle users; and They are the step electricity when connected to the grid and when off the grid respectively;

[0052] The above analysis can lead to the dispatch-oriented virtual battery model of electric vehicle aggregators:

[0053]

[0054] in, charging and discharging power for electric vehicle aggregators; for electric vehicle aggregators; is the step energy of the EV aggregator; P i eva,min and P i eva,max Charging power minimum and maximum values ​​for EV aggregators; and Provide upper and lower limits for electric vehicle aggregators;

[0055] Similarly, an air conditioner aggregator is an aggregate of all air conditioners in a county microgrid.

[0056] Assume that the upper and lower limits of indoor temperature comfort for air-conditioning users are [T min ,T max ], the state of charge of the air conditioning system can be defined as:

[0057]

[0058] According to the first-order thermal parameter model of air conditioning and the conversion of electric cooling power, the relationship between the state of charge and the charging and discharging power of the variable frequency air conditioning system can be obtained:

[0059]

[0060] in, is the user's indoor temperature; ac Air conditioning and refrigeration related parameters; and They are air conditioning power and floating charge power. In the variable frequency air conditioning thermal battery model, the floating charge power is defined as the electric power consumed by the variable frequency air conditioning system when maintaining the indoor temperature unchanged:

[0061]

[0062] Among them, T k outis the outdoor temperature; Q′ is the heat dissipation of indoor equipment, etc.; a, b, c, d are floating charge power related parameters;

[0063] Similarly, the Minkowski addition method is used to obtain the upper and lower boundaries of the dispatchable capacity of the air-conditioning aggregate load storage for the variable frequency air-conditioning charging and discharging power boundary, so as to reduce the dimension of the decision variables.

[0064]

[0065]

[0066] The above analysis can lead to the dispatch-oriented virtual battery model of air-conditioning aggregators:

[0067]

[0068] in, Electric power for air conditioning aggregators; Float charging power for air conditioning aggregators; is the virtual state of charge of the air conditioning aggregator; P i acs,min and P i acs,max It is the upper and lower limits of power for air conditioning aggregators; is the state of charge at a set temperature.

[0069] The technical solution is further optimized, the end-to-end local market transaction mechanism based on auction theory in step 1.3, specifically the following steps:

[0070] In order to encourage microgrids to participate in local P2P transactions, auction theory is introduced to construct a microgrid P2P energy transaction pricing model. The bidding auction between microgrids relies on a "P2P transaction platform", which is responsible for collecting bidding information, transaction allocation and pricing. Each microgrid reports the energy transaction volume and transaction bid to the platform. The platform matches the supply information and demand information according to the given transaction rules, determines the auction results and sends them to each microgrid.

[0071] For transaction period k, microgrid i participating in market transactions will declare its expected transaction power e i,k and transaction price λ i,k According to the amount of electricity traded by each microgrid, its market participants can be divided into buyers or sellers. i,k > 0 In microgrid i, the transaction role in time period k is a buyer, otherwise it is a seller. After the transaction roles are determined, a buyer set is formed Collect with sellers like Figure 4 As shown, buyers are sorted in descending order according to the prices they submitted. Sellers are sorted in ascending order according to the prices submitted. When the buying and selling sequence is formed, it enters the matching and clearing phase, that is, matching starts from the first buyer and seller in the set in turn, until the buying price is less than the selling price or exceeds the length of the buying and selling sequence, and the auction ends. When the buyer and seller are matched, the intermediate pricing method is used to calculate the transaction price between the matched buyer and seller, and the transaction quantity is equal to the lower value of the bid and ask quantity. After the auction ends, the remaining part of the microgrid transaction will be traded directly with the upper power grid.

[0072] If the electricity transaction price between microgrids λ bs,k Lower than the retail price of the distribution network Then the microgrid MG i with power shortage is more likely to be bs,k The price of λ is used to purchase electricity from the power-rich microgrid j, so that the operating cost of the microgrid can be reduced. bs,k Higher than the grid-connected electricity price of the distribution network Then microgrid j is more inclined to bs,k The surplus power is sold to the microgrid MG i with power shortage at a price of Therefore, in order to meet the The final clearing transaction price obtained under the declared electricity price restriction is bs,k , which is more conducive to promoting transactions between microgrid i with power shortage and microgrid j with power abundance, thereby reducing the operating costs of each other.

[0073] This technical solution is further optimized, and the integrated energy microgrid in step 1.4 participates in the dispatch optimization model of the local market, and the specific steps are:

[0074] In the P2P transaction mode, the microgrid energy management center usually arranges the scheduling plan of internal production equipment, storage equipment and flexible load, as well as the P2P transaction plan with other microgrids with the goal of maximizing its own power supply cost and transaction revenue; the scheduling optimization objective function of microgrid i is:

[0075]

[0076] Among them, R i is the total operating cost of microgrid i; The maintenance cost of production equipment and storage equipment for time period k; and Incentive compensation costs for EV aggregators and AC aggregators; Transaction costs for participating in the P2P market; The transaction cost with the main network;

[0077]

[0078] In the formula, and is the unit power maintenance cost coefficient and output of equipment l; Ω i is the equipment set of microgrid i; Δk is the time interval;

[0079]

[0080] In the formula, Unit compensation cost coefficients to incentivize EV aggregators to discharge for microgrid control; power for EV aggregators;

[0081]

[0082] In the formula, The compensation cost coefficient after the microgrid adjusts the air conditioner to aggregate commercial electric power; Set the indoor temperature value for the user;

[0083]

[0084] In the formula, is the power of interaction between microgrid and grid; and They are grid-connected price and retail price respectively;

[0085]

[0086] In the formula, The volume of natural gas purchased for the microgrid; ρ gas is the natural gas price of the gas grid;

[0087]

[0088] In the formula, λ ij,k and e ij,k are the transaction price and transaction volume between microgrid i and microgrid j respectively.

[0089] The technical solution is further optimized. In step 3, since the initial value of the model weight is randomly set at the beginning of training, if a larger learning rate is used, the model may fluctuate during the optimization process, affecting its convergence stability. The cosine annealing method is used to adjust the learning rate.

[0090] The technical solution is further optimized. In step 3, MATD3 is used to implement iterative update and optimization of the target strategy network, evaluation network 1 and evaluation network 2. The specific steps are as follows:

[0091] Step 3.1: Initialize the strategic network parameters θ of each microgrid ii , evaluate network parameters w i,1 and w i,2 And the corresponding target network parameter θ i '、w i ' ,1 and w i ' ,2 , experience pool D, sampling batch m, soft update coefficient τ, discount factor γ, update frequency C of the evaluation network relative to the policy network, Gaussian noise σ, decision steps K, current training times T cur , the maximum number of training times T max , the upper and lower limits of the parameters θ i,min / max and w i,min / max wait

[0092] Step 3.2, initialization decision time k = 0, initialize the interconnected microgrid system state, process the system operation data through the energy management center, and store the generated samples in the experience pool:

[0093] Step 3.2.1: Each microgrid i is based on the observed state o i,k , perform the action Get the corresponding reward value r i,k and the state o at the next moment i,k+1 , the sample data (s k ,a k ,r k ,s k+1 ) is stored in the experience pool;

[0094] Step 3.2.2, set k = k + 1, if k < K, return to step 3.2.1, if k = K, execute step 3.3;

[0095] Step 3.3, randomly select a batch from the experience pool to obtain m number of sample data from the experience pool D;

[0096] Step 3.4, calculate the loss function of each agent and minimize the loss function:

[0097]

[0098] And update the evaluation network parameters through cosine annealing:

[0099]

[0100] Step 3.5: Update the agent's policy network by gradient descent every C steps:

[0101]

[0102] And update the policy network parameters through cosine annealing:

[0103]

[0104] Step 3.6: Update the target network parameters of each agent using the soft update formula:

[0105] w i ' ,1 =τw i,1 +(1-τ)w i ' ,1

[0106] w i ' ,2 =τw i,1 +(1-τ)w i ' ,2

[0107] θ i '=τθ i +(1-τ)θ i '

[0108] If step<STEP, return to step 3.2; if step=STEP, stop learning and the network training is completed.

[0109] Different from the prior art, the above technical solution has the following beneficial effects:

[0110] 1. While coordinating and optimizing the interconnected microgrids, the present invention takes into account the randomness of renewable energy generation and user electric heating and cooling loads, as well as the impact of user-side demand response such as electric vehicles and variable-frequency air conditioners, and establishes a dispatchable capacity model for electric vehicles and air conditioning aggregators within the microgrid.

[0111] 2. The MATD3 selected by the present invention is an effective multi-agent reinforcement learning algorithm. It adopts the method of "centralized training and distributed execution" and can be applied to unstable environments. It can be applied to cooperative, competitive or mixed cooperative and competitive multi-agent environments. In addition, the nonlinear mapping ability of deep dense neural networks is utilized to incorporate the original input into each hidden layer of the network to enhance the nonlinear ability of the evaluation network, and the algorithm learning rate is adjusted by the cosine annealing method to improve the convergence rate and convergence threshold of the algorithm.

[0112] 3. The present invention can achieve coordinated optimization of energy between different counties, and further improve the supply and demand balance optimization of a single microgrid through power interaction between counties. In order to encourage energy trading activities between microgrids in the market, a P2P trading platform is established, and an auction-based pricing method is adopted to attract microgrids to participate in local transactions, thereby improving the overall economic efficiency of operation. In view of the random characteristics of different microgrids in counties, microgrids can complement power transactions through adjustable resources, enhance decision-making flexibility, serve as backup for each other, and improve system stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0113] Figure 1 It is the flow chart of energy management and transaction control of integrated energy microgrid;

[0114] Figure 2 It is a block diagram of energy trading in integrated energy microgrid;

[0115] Figure 3 It is a schematic diagram of energy flow in integrated energy microgrid;

[0116] Figure 4 Run algorithm framework for MATD3-based interconnected microgrids. DETAILED DESCRIPTION

[0117] In order to explain the technical content, structural features, achieved objectives and effects of the technical solution in detail, the following is a detailed description in conjunction with specific embodiments and accompanying drawings.

[0118] Example 1

[0119] See also Figure 2As shown in the figure, it is a block diagram of energy trading in integrated energy microgrids. These microgrids are connected to each other and to the power grid through lines. Each microgrid is equipped with an energy management system (EMS), which is responsible for the scheduling tasks within the microgrid and the P2P power trading tasks between microgrids. During the scheduling cycle of a day, each microgrid performs internal scheduling based on a decentralized and autonomous manner, and schedules its internal energy equipment according to the forecast of renewable energy power generation and load. Since the renewable energy power generation and load demand within each microgrid are different, if there is excess power within the microgrid, it can be sold to other microgrids through P2P transactions to earn profits. The introduction of the P2P model can provide a more flexible electricity price strategy than the power grid, and has a positive effect on guiding distributed energy supply systems to participate in the competition in the power market. After obtaining the internal scheduling plan, each microgrid EMS determines the role in the market during the trading period, declares the power and electricity price according to its own power surplus and shortage situation, and reports it to the P2P trading platform. The P2P trading platform matches and clears according to the information reported by each microgrid, prompting the microgrid to reach a transaction. If there is still some microgrid with surplus electricity that has not been traded, it will trade directly with the distribution network. The interconnected microgrid system includes three types of integrated energy microgrids with different load types: industrial, commercial and residential.

[0120] See Figure 1 and Figure 3 As shown, a method for energy management and trading of interconnected microgrids under user demand response includes the following steps:

[0121] Step 1: During the dispatch cycle of a day, the microgrid energy management center usually aims to minimize its own power supply cost and maximize transaction revenue, and establishes an operation optimization model for the integrated energy microgrid to participate in local market transactions:

[0122] Step 1.1, establish a scheduling model for energy production equipment and energy storage equipment in a comprehensive energy microgrid;

[0123] A thermal power network is established within each microgrid, in which the thermal network includes gas boilers, waste heat boilers, electric heat pumps, thermal energy storage and thermal loads, and the power network includes micro gas turbines, renewable energy, electric energy storage, flexible loads (such as electric vehicles, air conditioners, etc.) and rigid electric loads.

[0124] Gas turbine operating constraints:

[0125]

[0126] in, and are the electric power and thermal power generated by the gas turbine in the i-th microgrid during the k-th period, respectively; The natural gas power consumed by the gas turbine; is the calorific value of natural gas; and is the power generation efficiency and heat generation efficiency of the gas turbine; P i gt,min and P i gt,max The upper and lower limits of the gas turbine output electric power;

[0127] Gas boiler operation constraints:

[0128]

[0129] in, The thermal power generated by the gas boiler; The natural gas power consumed by the gas boiler; is the calorific value of natural gas; is the heat generation efficiency of the gas boiler; and The upper and lower limits of the heating power of the gas boiler;

[0130] Waste heat boiler operation constraints:

[0131]

[0132] in, is the thermal power generated by the waste heat boiler in the i-th microgrid during the k-time period; The thermal power absorbed by the waste heat boiler; is the heat generation efficiency of the gas boiler; and The upper and lower limits of the heat generation capacity of the waste heat boiler;

[0133] Electric heat pump operation constraints:

[0134]

[0135] in, is the heating power of the electric heat pump; P i eb The electrical power consumed by the electric heat pump; is the heating efficiency of the electric heat pump; and The upper and lower limits of the heat generation power of the electric heat pump;

[0136] Thermal storage tank operation constraints:

[0137]

[0138] in, They represent the power of the heat storage tank, respectively. A positive value indicates heat release, a negative value indicates heat storage, and a value of 0 indicates idleness. is the rated capacity of the heat storage tank; and Indicates the maximum heat storage and release power of the heat storage tank; is the heat storage state of the heat storage tank; Indicates the heat storage and release coefficient of the heat storage tank; and The upper and lower limits of the heat storage tank state;

[0139] Battery operation constraints:

[0140]

[0141] in, is the power of the battery, a positive value indicates discharge, a negative value indicates charging, and 0 indicates idle; is the rated capacity of the electric energy storage; P i ch,max and P i ds,max Indicates the maximum charging and discharging power of the heat storage tank; is the state of charge of the electrical energy storage; Indicates the charge and discharge coefficient of the battery; and The upper and lower limits of the battery status.

[0142] Step 1.2, establish a flexible load response characteristic model within the integrated energy microgrid;

[0143] The electric vehicle aggregator is the aggregation of all charging piles in a county microgrid, responsible for managing the charging and discharging scheduling of the electric vehicle cluster in the microgrid; the electric vehicle aggregator is composed of independent electric vehicles, and the power of the electric vehicles participating in the scheduling satisfies the following formula:

[0144]

[0145] in, and are the electric quantity and electric power of the electric vehicle respectively; and are the maximum and minimum power values ​​of the nth vehicle respectively; and They represent the minimum and maximum power of electric vehicles based on the charging and discharging efficiency of the charging pile; η ev The charging and discharging efficiency of electric vehicles; comprehensively consider the information of charging piles, car owners and EVs, and establish the upper and lower limits of the power when a single electric vehicle participates in scheduling;

[0146] The Minkowski addition method is used to obtain the upper and lower boundaries of the EV cluster dispatchability capacity for the electric vehicle charging and discharging power boundary and the battery safety power boundary to reduce the dimension of the decision variables.

[0147]

[0148] Among them, P i eva,max / min and They are the upper and lower limits of power and the upper and lower limits of electricity for electric vehicle aggregators; is a {0,1} variable, indicating the off-grid status of the electric vehicle;

[0149] The power of the electric vehicle aggregator includes the power of all grid-connected electric vehicles, but the off-grid behavior of electric vehicles will affect the power of the cluster. In order to solve the instantaneous step power phenomenon, the dynamic power of the EV cluster needs to be corrected;

[0150]

[0151] in, To correct the electrical quantity, The expected off-grid power consumption of electric vehicle users; and They are the step electricity when connected to the grid and when off the grid respectively;

[0152] The above analysis can lead to the dispatch-oriented virtual battery model of electric vehicle aggregators:

[0153]

[0154] in, charging and discharging power for electric vehicle aggregators; for electric vehicle aggregators; is the step energy of the EV aggregator; P i eva,min and P i eva,max Charging power minimum and maximum values ​​for EV aggregators; and Provide upper and lower limits for electric vehicle aggregators;

[0155] Similarly, an air conditioner aggregator is an aggregate of all air conditioners in a county microgrid.

[0156] Assume that the upper and lower limits of indoor temperature comfort for air-conditioning users are [T min ,T max ], the state of charge of the air conditioning system can be defined as:

[0157]

[0158] According to the first-order thermal parameter model of air conditioning and the conversion of electric cooling power, the relationship between the state of charge and the charging and discharging power of the variable frequency air conditioning system can be obtained:

[0159]

[0160] in, is the user's indoor temperature; ac Air conditioning and refrigeration related parameters; and are the air conditioning power and floating charge power respectively. In the variable frequency air conditioning thermal battery model, the floating charge power is defined as the electric power consumed by the variable frequency air conditioning system when maintaining the indoor temperature unchanged;

[0161]

[0162] Among them, T k out is the outdoor temperature; Q′ is the heat dissipation of indoor equipment, etc.; a, b, c, d are floating charge power related parameters;

[0163] Similarly, the Minkowski addition method is used to obtain the upper and lower boundaries of the dispatchable capacity of the air-conditioning aggregate load storage for the variable frequency air-conditioning charging and discharging power boundary, so as to reduce the dimension of the decision variables.

[0164]

[0165] The above analysis can lead to the dispatch-oriented virtual battery model of air-conditioning aggregators:

[0166]

[0167] in, Electric power for air conditioning aggregators; Float charging power for air conditioning aggregators; is the virtual state of charge of the air conditioning aggregator; P i acs,min and P i acs,max It is the upper and lower limits of power for air conditioning aggregators; is the state of charge at a set temperature.

[0168] Step 1.3: Establish an end-to-end local market transaction mechanism based on auction theory

[0169] Considering the energy exchange and interest interaction between microgrids, it is necessary to design a reasonable transaction clearing mechanism to improve the economy and reliability within the county. In order to encourage microgrids to participate in local P2P transactions, the auction theory is introduced to construct a microgrid P2P energy transaction pricing model. The bidding auction between microgrids relies on a "P2P trading platform", which is responsible for collecting bidding information, transaction allocation and pricing. Each microgrid reports the energy transaction volume and transaction bid to the platform. The platform matches the supply information and demand information according to the given transaction rules, determines the auction results and sends them to each microgrid.

[0170] In t k At time t, microgrid i participating in market transactions will declare its expected transaction power e i,k and transaction price λ i,k According to the amount of electricity traded by each microgrid, its market participants can be divided into buyers or sellers. i,k > 0 In microgrid i, the transaction role in time period k is a buyer, otherwise it is a seller. After the transaction roles are determined, a buyer set is formed Collect with sellers like Figure 4 As shown, buyers are sorted in descending order according to the prices they submitted. Sellers are sorted in ascending order according to the prices submitted. When the buying and selling sequence is formed, it enters the matching and clearing phase, that is, matching starts from the first buyer and seller in the set in turn, until the buying price is less than the selling price or exceeds the length of the buying and selling sequence, and the auction ends. When the buyer and seller are matched, the intermediate pricing method is used to calculate the transaction price between the matched buyer and seller, and the transaction quantity is equal to the lower value of the bid and ask quantity. After the auction ends, the remaining part of the microgrid transaction will be traded directly with the upper power grid.

[0171] If the electricity transaction price between microgrids λ bs,k Lower than the retail price of the distribution network Then the microgrid i with power shortage is more likely to be bs,k The price of λ is used to purchase electricity from the power-rich microgrid j, so that the operating cost of the microgrid can be reduced. bs,k Higher than the grid-connected electricity price of the distribution network Then j prefers to take λ bs,k The excess power is sold to the microgrid i with power shortage at a price of Therefore, in order to meet the The final clearing transaction price obtained under the declared electricity price restriction is bs,k , which is more conducive to promoting transactions between microgrid i with power shortage and microgrid j with power abundance, thereby reducing the operating costs of each other.

[0172] Step 1.4: Establish a dispatch optimization model for integrated energy microgrids to participate in the local market;

[0173] In the P2P transaction mode, the microgrid energy management center usually arranges the scheduling plan of internal production equipment, storage equipment and flexible load, as well as the P2P transaction plan with other microgrids with the goal of maximizing its own power supply cost and transaction revenue; the scheduling optimization objective function of microgrid i is:

[0174]

[0175] Among them, R i is the total operating cost of microgrid i; The maintenance cost of production equipment and storage equipment for time period k; and Incentive compensation costs for EV aggregators and AC aggregators; Transaction costs for participating in the P2P market; The transaction cost with the main network;

[0176]

[0177] In the formula, and is the unit power maintenance cost coefficient and output of equipment l; Ω i is the equipment set of microgrid i; Δk is the time interval;

[0178]

[0179] In the formula, Unit compensation cost coefficients to incentivize EV aggregators to discharge for microgrid control; Power for EV Aggregators

[0180]

[0181] In the formula, The compensation cost coefficient after the microgrid adjusts the air conditioner to aggregate commercial electric power; Set the indoor temperature value for the user;

[0182]

[0183] In the formula, is the power of interaction between microgrid and grid; and They are grid-connected price and retail price respectively;

[0184]

[0185] In the formula, The volume of natural gas purchased for the microgrid; ρ gas is the natural gas price of the gas grid;

[0186]

[0187] In the formula, λ ij,k and e ij,k are the transaction price and transaction volume between microgrid i and microgrid j respectively.

[0188] Step 2: Build the deep neural network required for the Multi-Agent TwinDelayed Deep Deterministic Policy Gradient Algorithm (MATD3):

[0189] MATD3 is a classic multi-agent reinforcement learning algorithm that adopts the method of "centralized training and distributed execution". It is applicable to unstable environments and can be used in cooperative, competitive or mixed cooperative and competitive multi-agent environments. Each agent in MATD3 consists of a current policy network, a target policy network, two comment networks 1 and 2, and two target evaluation networks 1 and 2. The six neural networks are all fully connected neural networks, including input layers, hidden layers, and output layers. The current policy network and the target policy network have the same structure, and the evaluation network 1, evaluation network 2, target evaluation network 1 and target evaluation network 2 have the same structure. In addition, the nonlinear mapping ability of deep dense neural networks is used to incorporate the original input x into each hidden layer of the network, thereby retaining the information of the input x;

[0190] Step 3: Read Figure 4 ,According to the model established in step 1, the energy management center is used to interact with the interconnected integrated energy microgrid, historical interaction information is obtained and stored in the experience pool, and MATD3 is used to implement iterative update and optimization of the strategy network, evaluation network 1 and evaluation network 2;

[0191] Step 3.1: Initialize the strategic network parameters θ of each microgrid i i , evaluate network parameters w i,1 and w i,2 And the corresponding target network parameter θ i '、w i ' ,1 and w i ' ,2 , experience pool D, sampling batch m, soft update coefficient τ, discount factor γ, update frequency C of the evaluation network relative to the policy network, Gaussian noise σ, decision steps K, current training times T cur , the maximum number of training times T max , the upper and lower limits of the parameters θ i,min / max and w i,min / max wait

[0192] Step 3.2, initialization decision time k = 0, initialize the interconnected microgrid system state, process the system operation data through the energy management center, and store the generated samples in the experience pool:

[0193] Step 3.2.1: Each microgrid i is based on the observed state o i,k, perform the action Get the corresponding reward value r i,k and the state o at the next moment i,k+1 , the sample data (s k ,a k ,r k ,s k+1 ) is stored in the experience pool;

[0194] Step 3.2.2, set k = k + 1, if k < K, return to step 3.2.1, if k = K, execute step 3.3;

[0195] Step 3.3, randomly select a batch from the experience pool to obtain m number of sample data from the experience pool D;

[0196] Step 3.4, calculate the loss function of each agent and minimize the loss function:

[0197]

[0198] And update the evaluation network parameters through cosine annealing:

[0199]

[0200] Step 3.5: Update the agent's policy network by gradient descent every C steps:

[0201]

[0202] And update the policy network parameters through cosine annealing:

[0203]

[0204] Step 3.6: Use the soft update formula to update the target network parameters of each agent:

[0205] w i ' ,1 =τw i,1 +(1-τ)w i ' ,1

[0206] w i ' ,2 =τw i,1 +(1-τ)w i ' ,2

[0207] θ i '=τθ i +(1-τ)θ i '

[0208] If step<STEP, return to step 3.2; if step=STEP, stop learning and the network training is completed.

[0209] Step 4: Based on the strategy network obtained in step 3, energy management and energy trading of the integrated energy microgrid are realized:

[0210] At any decision-making moment in the operation of the system, the operating status of the integrated energy microgrid, the electricity trading volume, the electricity trading quotation and other information are normalized and input into the strategy network of MATD3. After forward propagation, the action information output by the strategy network is denormalized to obtain the actions that the system can take at the current decision moment. The operating cost, transaction income and incentive cost obtained by the integrated energy microgrid after repeatedly performing the above operations in a scheduling cycle are observed.

[0211] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "include..." or "comprise..." do not exclude the existence of other elements in the process, method, article or terminal device including the elements. In addition, in this article, "greater than", "less than", "exceed" and the like are understood to exclude the number itself; "above", "below", "within" and the like are understood to include the number itself.

[0212] Although the above embodiments have been described, once those skilled in the art know the basic creative concepts, they can make additional changes and modifications to these embodiments. Therefore, the above description is only an embodiment of the present invention and does not limit the patent protection scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the specification and drawings of the present invention, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for energy management and trading of interconnected microgrids under user demand response, characterized in that: The interconnected microgrid system includes three types of integrated energy microgrids with different load types: industrial, commercial and residential. The integrated energy microgrid includes an energy management center, energy production equipment, energy storage equipment and flexible loads. The energy production equipment includes gas turbines, waste heat boilers, gas boilers, electric heat pumps, and new energy power generation devices. The energy storage equipment includes batteries and heat storage tanks. The flexible loads include electric vehicle load aggregators and air conditioning load aggregators. The energy management center is an energy and information integration center. It undertakes gas suppliers, distribution network suppliers and end-to-end trading platforms externally, and can obtain information such as electricity prices, natural gas prices and clearing plans issued by trading platforms, control the purchase and trading volume of natural gas and electricity, and can issue response invitations to flexible loads internally to adjust user needs and control the production plan of energy equipment in the system. The energy management and trading method specifically includes the following steps: Step 1: In the end-to-end transaction mode, the energy management center usually aims to minimize its own power supply cost and maximize transaction revenue, and establishes an operation optimization model for the integrated energy microgrid to participate in local end-to-end market transactions; Step 1.1, establish a scheduling model for energy production equipment and energy storage equipment in a comprehensive energy microgrid; Step 1.2, establish the load response characteristic model of electric vehicles and variable frequency air conditioners in the integrated energy microgrid; Step 1.3: Establish an end-to-end local market transaction mechanism based on auction theory; Step 1.4: Establish a dispatch optimization model and process for integrated energy microgrids to participate in the local end-to-end market; Step 2: Construct the deep neural network required for the multi-agent dual-delay deep deterministic policy gradient algorithm: Each agent in the multi-agent double-delay deep deterministic policy gradient algorithm is composed of a current policy network, a target policy network, evaluation network 1 and evaluation network 2, and target evaluation network 1 and target evaluation network 2. The six neural networks are all fully connected neural networks, including an input layer, a hidden layer, and an output layer. The current policy network and the target policy network have the same structure, and the evaluation network 1, the evaluation network 2, the target evaluation network 1 and the target evaluation network 2 have the same structure. In addition, each layer of the network uses the nonlinear mapping ability of the deep dense neural network to incorporate the original input x into each hidden layer of the network, thereby retaining the information of the input x; Step 3: Based on the model established in step 1, the energy management center is used to interact with the interconnected integrated energy microgrid simulation environment to obtain historical information and store it in the experience pool, and the multi-agent double-delay deep deterministic policy gradient algorithm is used to iteratively update and optimize the target policy network, evaluation network 1, and evaluation network 2; Step 4: Implement energy management and energy trading solutions for the integrated energy microgrid based on the strategy network obtained in step 3: At any decision-making moment in the operation of the system, the decision-making moment, the operating status of the integrated energy microgrid, the electricity trading volume and electricity trading quotation and other information are normalized and input into the policy network of the multi-agent double-delay deep deterministic policy gradient algorithm. After forward propagation, the action information output by the policy network is denormalized to obtain the actions that the system can take at the current decision moment. The operating cost, transaction income and incentive cost obtained by the integrated energy microgrid after repeatedly performing the above operations in a scheduling cycle are observed.

2. The interconnected microgrid energy management and trading method under user demand response according to claim 1 is characterized in that: In step 1.1, a scheduling model for energy production equipment and energy storage equipment of a comprehensive energy microgrid is established, and the specific steps are as follows: A thermal power network is built inside each microgrid, where the thermal network includes gas boilers, waste heat boilers, electric heat pumps, thermal energy storage and thermal loads, and the power network includes micro gas turbines, renewable energy, electric energy storage, flexible loads and rigid electric loads; Gas turbine operating constraints: in, and are the electric power and thermal power generated by the gas turbine in the i-th microgrid during the k-th period, respectively; The natural gas power consumed by the gas turbine; is the calorific value of natural gas; and The power generation efficiency and heat generation efficiency of the gas turbine; and The upper and lower limits of the gas turbine output electric power; Gas boiler operation constraints: in, The thermal power generated by the gas boiler; The natural gas power consumed by the gas boiler; is the calorific value of natural gas; is the heat generation efficiency of the gas boiler; and The upper and lower limits of the heating power of the gas boiler; Waste heat boiler operation constraints: in, is the thermal power generated by the waste heat boiler in the i-th microgrid during the k-time period; The thermal power absorbed by the waste heat boiler; is the heat generation efficiency of the gas boiler; and The upper and lower limits of the heat generation capacity of the waste heat boiler; Electric heat pump operation constraints: in, is the heating power of the electric heat pump; The electrical power consumed by the electric heat pump; is the heating efficiency of the electric heat pump; and The upper and lower limits of the heat generation power of the electric heat pump; Thermal storage tank operation constraints: in, They represent the power of the heat storage tank, respectively. A positive value indicates heat release, a negative value indicates heat storage, and a value of 0 indicates idleness. is the rated capacity of the heat storage tank; and Indicates the maximum heat storage and release power of the heat storage tank; is the heat storage state of the heat storage tank; Indicates the heat storage and release coefficient of the heat storage tank; and The upper and lower limits of the heat storage tank state; Battery operation constraints: in, is the power of the battery, a positive value indicates discharge, a negative value indicates charging, and 0 indicates idle; is the rated capacity of the electric energy storage; and Indicates the maximum charging and discharging power of the heat storage tank; is the state of charge of the electrical energy storage; Indicates the charge and discharge coefficient of the battery; and The upper and lower limits of the battery status.

3. The interconnected microgrid energy management and trading method under user demand response according to claim 2 is characterized in that: The response characteristic model of the electric vehicle aggregator and the variable frequency air conditioner aggregator within the integrated energy microgrid in step 1.2 is specifically as follows: The electric vehicle aggregator is the aggregation of all charging piles in a county microgrid, responsible for managing the charging and discharging scheduling of the electric vehicle cluster in the microgrid; the electric vehicle aggregator is composed of independent electric vehicles, and the power of the electric vehicles participating in the scheduling satisfies the following formula: in, and are the electric quantity and electric power of the electric vehicle respectively; and are the maximum and minimum power values ​​of the nth vehicle respectively; and They represent the minimum and maximum power of electric vehicles based on the charging and discharging efficiency of the charging pile; η ev The charging and discharging efficiency of electric vehicles; comprehensively consider the information of charging piles, car owners and electric vehicles, and establish the upper and lower limits of the power when a single electric vehicle participates in scheduling; The Minkowski addition method is used to obtain the upper and lower boundaries of the dispatchable capacity of electric vehicle aggregators for the electric vehicle charging and discharging power boundaries and the battery safety power boundaries, so as to reduce the dimension of decision variables; in, and They are the upper and lower limits of power and the upper and lower limits of electricity for electric vehicle aggregators; is a {0,1} variable, indicating the off-grid status of the electric vehicle; The power of the electric vehicle aggregator includes the power of all grid-connected electric vehicles, but the off-grid behavior of electric vehicles will affect the power of the cluster. In order to solve the instantaneous step power phenomenon, the dynamic power of the electric vehicle aggregator needs to be corrected; in, To correct the electrical quantity, The expected off-grid power consumption of electric vehicle users; and They are the step electricity when connected to the grid and when off the grid respectively; The above analysis can lead to the dispatch-oriented virtual battery model of electric vehicle aggregators: in, charging and discharging power for electric vehicle aggregators; for electric vehicle aggregators; step energy for EV aggregators; and Charging power minimum and maximum values ​​for EV aggregators; and Provide upper and lower limits for electric vehicle aggregators; Similarly, an air conditioner aggregator is an aggregate of all air conditioners in a county microgrid; Assume that the upper and lower limits of indoor temperature comfort for air-conditioning users are [T min ,T max ], the state of charge of the air conditioning system can be defined as: According to the first-order thermal parameter model of air conditioning and the conversion of electric cooling power, the relationship between the state of charge and the charging and discharging power of the variable frequency air conditioning system can be obtained: in, is the user's indoor temperature; ac Air conditioning and refrigeration related parameters; and are the air conditioning power and floating charge power respectively. In the variable frequency air conditioning thermal battery model, the floating charge power is defined as the electric power consumed by the variable frequency air conditioning system when maintaining the indoor temperature unchanged; Among them, T k out is the outdoor temperature; Q′ is the heat dissipation of indoor equipment, etc.; a, b, c, d are floating charge power related parameters; Similarly, the Minkowski addition method is used to obtain the upper and lower boundaries of the dispatchable capacity of the air-conditioning aggregate load storage for the variable frequency air-conditioning charging and discharging power boundary, so as to reduce the dimension of the decision variables. The above analysis can lead to the dispatch-oriented virtual battery model of air-conditioning aggregators: in, Electric power for air conditioning aggregators; Float charging power for air conditioning aggregators; Virtual state of charge for air conditioning aggregators; and It is the upper and lower limits of power for air conditioning aggregators; is the state of charge at a set temperature.

4. The interconnected microgrid energy management and trading method under user demand response according to claim 2 is characterized in that: The end-to-end local market transaction mechanism based on auction theory in step 1.3 has the following specific steps: In order to encourage microgrids to participate in local P2P transactions, auction theory is introduced to construct a microgrid P2P energy transaction pricing model; the bidding auction between microgrids relies on a "P2P transaction platform", which is responsible for collecting bidding information, transaction allocation and pricing. Each microgrid reports the energy transaction volume and transaction bid to the platform. The platform matches the supply information and demand information according to the given transaction rules, determines the auction results and sends them to each microgrid; For transaction period k, microgrid i participating in market transactions will declare its expected transaction power e i,k and transaction price λ i,k According to the transaction amount of each microgrid, its market participants can be divided into buyers or sellers. i,k > 0 In the transaction role of microgrid i in time period k, it is a buyer, otherwise it is a seller; when all microgrids After the transaction roles are determined, a buyer set is formed Collect with sellers Buyers are sorted in descending order according to the prices they submitted. Sellers are sorted in ascending order according to the prices submitted. When the buying and selling sequence is formed, it enters the matching and clearing stage, that is, matching starts from the first buyer and seller in the set in turn, until the buying price is less than the selling price or exceeds the length of the buying and selling sequence, and the auction ends; when the buyer and seller are matched, the intermediate pricing method is used to calculate the transaction price between the matched buyer and seller, and the transaction quantity is equal to the lower value of the bid and ask quantity; after the auction ends, the remaining part of the microgrid transaction will be traded directly with the upper-level power grid; If the electricity transaction price between microgrids λ bs,k Lower than the retail price of the distribution network Then the microgrid MG i with power shortage is more likely to be bs,k The price of λ is used to purchase electricity from the power-rich microgrid j, so that the operating cost of the microgrid can be reduced; similarly, if λ bs,k Higher than the grid-connected electricity price of the distribution network Then microgrid j is more inclined to bs,k The excess power is sold to the microgrid i with power shortage at a price of The profit of selling to the distribution network at a price of The final clearing transaction price obtained under the declared electricity price restriction is bs,k , which is more conducive to promoting transactions between microgrid i with power shortage and microgrid j with power abundance, thereby reducing the operating costs of each other.

5. The interconnected microgrid energy management and trading method under user demand response according to claim 2 is characterized in that: The dispatch optimization model of the integrated energy microgrid participating in the local market in step 1.4 has the following specific steps: In the P2P transaction mode, the microgrid energy management center usually arranges the scheduling plan of internal production equipment, storage equipment and flexible load, as well as the P2P transaction plan with other microgrids with the goal of maximizing its own power supply cost and transaction revenue; the scheduling optimization objective function of microgrid i is: Among them, R i is the total operating cost of microgrid i; The maintenance cost of production equipment and storage equipment for time period k; and Incentive compensation costs for EV aggregators and AC aggregators; Transaction costs for participating in the P2P market; The transaction cost with the main network; In the formula, and is the unit power maintenance cost coefficient and output of equipment l; Ω i is the equipment set of microgrid i; Δk is the time interval; In the formula, Unit compensation cost coefficients to incentivize EV aggregators to discharge for microgrid control; power for electric vehicle aggregators; In the formula, The compensation cost coefficient after the microgrid adjusts the air conditioner to aggregate commercial electric power; Set the indoor temperature value for the user; In the formula, is the power of interaction between microgrid and grid; and They are grid-connected price and retail price respectively; In the formula, The volume of natural gas purchased for the microgrid; ρ gas is the natural gas price of the gas grid; In the formula, λ ij,k and e ij,k are the transaction price and transaction quantity between microgrid i and microgrid j respectively.

6. The interconnected microgrid energy management and trading method under user demand response according to claim 1 is characterized in that: In step 3, since the initial value of the model weight is randomly set at the beginning of training, if a larger learning rate is used, the model may fluctuate during the optimization process, affecting its convergence stability. The cosine annealing method is used to adjust the learning rate.

7. The interconnected microgrid energy management and trading method under user demand response according to claim 1 is characterized in that: The step 3 uses MATD3 to implement iterative update and optimization of the target strategy network, evaluation network 1 and evaluation network 2. The specific steps are: Step 3.1: Initialize the strategic network parameters θ of each microgrid i i , evaluate network parameters w i,1 and w i,2 And the corresponding target network parameter θ i '、w i ' ,1 and w i ' ,2 , experience pool D, sampling batch m, soft update coefficient τ, discount factor γ, update frequency C of the evaluation network relative to the policy network, Gaussian noise σ, decision steps K, current training times T cur , the maximum number of training times T max , the upper and lower limits of the parameters θ i,min / max and w i,min / max wait Step 3.2, initialization decision time k = 0, initialize the interconnected microgrid system state, process the system operation data through the energy management center, and store the generated samples in the experience pool: Step 3.2.1: Each microgrid i is based on the observed state o i,k , perform action a i,k =μ θi' (o i,k )+v,v=N(0,σ 2 ), and get the corresponding reward value r i,k and the state o at the next moment i,k+1 , the sample data (s k ,a k ,r k ,s k+1 ) is stored in the experience pool; Step 3.2.2, set k = k + 1, if k < K, return to step 3.2.1, if k = K, execute step 3.3; Step 3.3, randomly select a batch from the experience pool to obtain m number of sample data from the experience pool D; Step 3.4, calculate the loss function of each agent and minimize the loss function: And update the evaluation network parameters through cosine annealing: Step 3.5: Update the agent's policy network by gradient descent every C steps: And update the policy network parameters through cosine annealing: Step 3.6: Use the soft update formula to update the target network parameters of each agent: w i ' ,1 =τw i,1 +(1-τ)w i ' ,1 w i ' ,2 =τw i,1 +(1-τ)w i ' ,2 i i '=τθ i +(1-τ)θ i ' If step<STEP, return to step 3.2; if step=STEP, stop learning and the network training is completed.

Images