A micro-grid electricity-carbon combined transaction method, system and medium

Abstract

The application belongs to the technical field of power systems, and specifically discloses a micro-grid electricity-carbon combined transaction method, a system and a medium, which comprises the following steps: constructing a regional interconnected micro-grid system model; constructing a local electricity-carbon combined market mechanism, adopting a multi-round bilateral auction mode to perform order matching, combining the electricity price and the carbon price joint bidding rules to determine the transaction result and verify through the power flow constraint; based on the micro-grid model and the market mechanism, constructing a multi-agent Markov decision process model, setting a state space, an action space and a multi-objective reward function; adopting a multi-agent proximal policy optimization algorithm MAPPO-TDSA which fuses task decomposition and attention mechanism to train each agent policy network, and according to the micro-grid transaction execution result and the electricity carbon quota balance, respectively settling with the external electricity market and the carbon transaction market. The application realizes the energy-carbon collaborative transaction and intelligent regulation and control among regional micro-grids, and has significant low-carbon, autonomy and strategy optimization capabilities.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of power systems, and particularly relates to a micro-grid electricity-carbon combined transaction method, system and medium. BACKGROUND

[0002] With large-scale access of renewable energy and development of distributed power systems, regional interconnected micro-grids, as an important part of new power systems, have been widely applied in multi-source collaboration and flexible scheduling scenarios. Micro-grids have strong autonomy, good operation flexibility, and rich controllable resources, which provide an effective path for local consumption and flexible use of distributed clean energy.

[0003] However, the micro-grid scheduling methods in the prior art are generally limited to a single power optimization dimension, lack dynamic coupling of carbon emission factors, cannot reflect the real impact of carbon cost on resource allocation and operation strategy, are difficult to support the low-carbon operation goal of "source-load-carbon" collaboration, and generally rely on centralized control or global information acquisition in the market interaction process, which is difficult to realize local learning and adaptive optimization of strategies at the agent level.

[0004] Traditional scheduling methods are generally based on static models or linear optimization, which are difficult to adapt to the complex characteristics of new energy volatility, carbon price uncertainty, and interaction diversity among micro-grids. On the other hand, there is still a lack of localized electricity-carbon combined transaction mechanism suitable for micro-grids in the current regional range, which cannot support efficient circulation of energy and carbon elements among multiple micro-grids, resulting in low efficiency of carbon quota resource allocation and broken carbon emission reduction incentive chain. SUMMARY

[0005] The application provides a micro-grid electricity-carbon combined transaction method, system and medium to solve the problem of lack of carbon quota coupling and intelligent control ability of micro-grids in the prior art.

[0006] The technical solution adopted by the application is as follows:

[0007] In a first aspect, the application provides a micro-grid electricity-carbon combined transaction method, which includes the following steps:

[0008] Step S1, a regional interconnected micro-grid system model is constructed, each micro-grid is modeled as an agent with local load, renewable energy generation equipment, energy storage carbon fixation device and controllable gas turbine, and power output, carbon emission and operation cost models of each equipment are defined;

[0009] Step S2, a local electricity-carbon combined market mechanism is constructed, electricity flow and carbon emission flow are bound in the electricity-carbon combined market, multi-round bilateral auction is used to match buy and sell orders, and transaction results are determined according to the joint bidding rules of electricity price and carbon price, and transaction settlement is verified by power line flow constraints;

[0010] Step S3, based on the micro-grid model and the electricity-carbon market mechanism, a multi-agent Markov decision process model is constructed, and the state space, action space and multi-objective reward function are set;

[0011] Step S4, a multi-agent proximal policy optimization algorithm MAPPO-TDSA is used to train the strategy network of each micro-grid agent, and the optimal strategy for electricity-carbon joint market transaction and resource regulation is output;

[0012] Step S5, each micro-grid generates expected transaction and scheduling actions using the trained strategy network based on the observed state, and after passing the hard constraint test, completes the matching and clearing of transaction orders based on the multi-round bilateral auction mechanism, determines the electricity transaction volume, carbon quota transfer and corresponding settlement price, and executes the resource scheduling behavior;

[0013] Step S6, according to the micro-grid transaction execution results and the electricity and carbon quota balance, energy compensation and carbon quota settlement are respectively carried out with the external electricity market and carbon trading market, and the transaction feedback is used for the next round of strategy optimization.

[0014] Further, step S1 includes the following steps:

[0015] Step S1-1, define the local load, energy storage and carbon fixation device, controllable gas turbine and renewable energy generation equipment contained in each micro-grid, and establish the power output model and carbon emission model of each device;

[0016] Step S1-2, the energy storage and carbon fixation device adopts an integrated battery structure with dual functions of energy storage and carbon dioxide absorption, and sets the state of charge, charging and discharging efficiency, carbon fixation efficiency and maximum carbon fixation power constraint;

[0017] Step S1-3, the output power and operating cost of the controllable gas turbine form a nonlinear quadratic function relationship, its carbon emission is calculated according to the fuel carbon content factor, thermal efficiency and combustion oxidation rate, and has maximum power, minimum power and climbing rate constraints;

[0018] Step S1-4, the renewable energy equipment includes wind power generation module and photovoltaic power generation module;

[0019] The wind power generation module adopts an interval prediction modeling method, and builds an upper and lower limit output prediction interval based on historical wind speed data and meteorological factors;

[0020] The photovoltaic power generation module adopts a point prediction method, and calculates a single output value using the sunshine intensity and radiation model.

[0021] Further, step S2 includes the following steps:

[0022] Step S2-1, a local electricity-carbon combined market platform is established, and the microgrid is regarded as a market participant, and the market transaction object includes green electricity and corresponding carbon quota;

[0023] Step S2-2, a combined bidding rule is set, each microgrid acts as a buyer or a seller to submit a power bid and a carbon quota bid, and the bid is dynamically generated in combination with the predicted green power supply and load demand;

[0024] Step S2-3, a multi-round bilateral auction mechanism is used for order matching, the buy-side bid is arranged in descending order and the sell-side bid is arranged in ascending order, the highest buy price and the lowest sell price are compared in turn, and if the condition is met, the matching is achieved;

[0025] Step S2-4, in each round of transaction, the weighted average method is used to calculate the settlement price according to the matching amount and the market supply-demand ratio, and the final transaction settlement amount is determined by the electricity price and the carbon price.

[0026] Further, step S2-3 includes the following steps:

[0027] Step S2-3-1, a buyer order pool and a seller order pool are constructed according to the expected order set, and are sorted according to the bid in descending order and ascending order respectively;

[0028] Step S2-3-2, the order pool after sorting is executed for round-by-round order matching operation, the current highest buy-side bid and the lowest sell-side bid are compared in each round, if the buy price is not lower than the sell price, it is determined as a tradable order pair, and the traded electricity quantity is the minimum remaining quantity of the buy-side and the sell-side;

[0029] Step S2-3-3, after each round of order matching, based on the physical topology structure and the line transmission capacity of the microgrid interconnection, the power flow safety check is executed, and whether the actual power transmission of each distribution line meets the capacity limit is calculated;

[0030] Step S2-3-4, if a round of matching transaction pair causes line overload, the transaction settlement of the matching pair is terminated, and the order pool is updated to enter the next round of matching.

[0031] Further, step S3 includes the following steps:

[0032] Step S3-1, the operation decision process of each microgrid agent is modeled as a Markov decision process, and the state transition of the Markov decision process depends on the observation state at the current time, the action decision and the environmental feedback result;

[0033] Step S3-2, define a state space, an action space and a reward function triple for each microgrid agent, wherein the state space is composed of local observable variables, the action space is a continuous variable set, and the reward function is designed in combination with economic, environmental and constraint factors;

[0034] Step S3-3, a discrete-time model with a time step of 24 is used to divide the daily operation cycle into 24 time steps, and the reinforcement learning strategy is updated at each time step to obtain an optimal control strategy based on dynamic markets and operating environments;

[0035] Step S3-4, based on the round experience sampling mechanism in reinforcement learning, the state-action-reward-next state sequence data of the microgrid in the complete scheduling cycle is collected as the sample basis for training the policy network.

[0036] Further, in step S3-2:

[0037] The state space includes: the upper and lower limits of the predicted output of wind power generation, the predicted value of photovoltaic power generation, the predicted value of local user load, the state of charge of energy storage and carbon sequestration device, the market settlement price and carbon quota state of the last time step;

[0038] The action space is a continuous variable set, including: the expected output power of the controllable gas turbine, the expected charge and discharge power and carbon sequestration mode control parameters of the energy storage and carbon sequestration device, and the expected offer and expected traded electricity of the local electricity-carbon joint market;

[0039] The reward function includes economic reward items, environmental reward items, false transaction penalty items and scheduling out-of-boundary penalty items;

[0040] The economic reward item is calculated by the transaction revenue minus the operating cost, and the environmental reward item gives positive incentive according to the amount of carbon sequestration and negative punishment according to the amount of carbon emissions;

[0041] The reward function is used as the optimization target of the agent policy network, and the state-action sequence generated by trajectory sampling is used to update the backward gradient of the policy network.

[0042] Further, in step S4, the strategy network is trained using the multi-agent proximal policy optimization algorithm MAPPO-TDSA, wherein each agent maintains a local strategy network and critic network;

[0043] The optimization algorithm includes a task decomposition mechanism, which decomposes the overall reward function into multiple sub-task rewards, and establishes a sub-task value function for each;

[0044] The optimization algorithm includes an attention mechanism that dynamically adjusts the weight of each sub-task on joint strategy evaluation;

[0045] The policy updating is based on information shared by neighbor nodes in a communication topology, and a distributed training process is implemented through consensus updating rules.

[0046] Further, in step S5, each microgrid generates an expected transaction action according to the current observed state and performs a hard constraint check, including the upper limit of gas turbine power, the upper and lower limits of energy storage power, and the feasibility of carbon sequestration power.

[0047] After the check passes, the agent submits the transaction action to the electricity-carbon joint market and participates in multiple rounds of bilateral auctions to determine the actual transaction pair, the traded electricity quantity, the settlement price, and the carbon quota transfer.

[0048] The execution result is used as the input of the next round of state space and for training data expansion and model incremental optimization.

[0049] In a second aspect, the present application provides a microgrid electricity-carbon joint transaction system, which comprises:

[0050] A system modeling module is used to build a regional interconnected microgrid system model, model each microgrid as an agent with local load, renewable energy generation equipment, energy storage and carbon sequestration device, and controllable gas turbine, and define the power output, carbon emission and operation cost model of each device.

[0051] A market mechanism building module is used to build a local electricity-carbon joint market mechanism, bind electricity flow and carbon emission flow in the market, set joint bidding rules and match buy and sell orders in multiple rounds of bilateral auctions, and the transaction settlement is subject to power flow constraint check.

[0052] A decision modeling module is used to build a multi-agent Markov decision process model based on the microgrid system model and the electricity-carbon market mechanism, define the state space, action space and multi-objective reward function of each agent.

[0053] A strategy training module is used to use a multi-agent proximal policy optimization algorithm MAPPO-TDSA that integrates task decomposition and attention mechanism to jointly train the strategy network of each agent and output the optimal strategy for joint transaction and resource regulation.

[0054] A strategy execution module is used to generate transaction and dispatch actions using the strategy network based on the current observed state, perform a hard constraint check, and then trade through the electricity-carbon joint market to complete electricity trading volume, carbon quota transfer and resource regulation.

[0055] An external interaction module is used to perform energy compensation and carbon quota settlement with external electricity markets and carbon trading markets based on the microgrid transaction execution result, and use the feedback information for strategy updating.

[0056] In a third aspect, the present application provides a computer readable storage medium, which stores computer instructions, and when a computer reads the computer instructions in the storage medium, the computer executes the micro-grid electricity-carbon combined transaction method according to the first aspect.

[0057] It can be seen from the above technical solutions that the present application has the following advantages:

[0058] The present application provides a micro-grid electricity-carbon combined transaction method, which builds a regional interconnected heterogeneous micro-grid system model and a local electricity-carbon combined market mechanism, opens up the linkage channel of electricity flow and carbon emission flow in the distributed system, and solves the problem of energy-carbon fragmentation in the existing micro-grid dispatching; introduces a storage energy carbon fixation device, a wind-solar prediction mechanism and a gas turbine carbon emission model, realizes the collaborative modeling of multiple types of resources and the multi-objective dispatching under carbon constraints; through a multi-agent modeling framework based on Markov decision process, combined with multi-objective reward design and MAPPO-TDSA deep reinforcement learning algorithm, the adaptive optimization ability of the micro-grid in an uncertain environment is effectively improved; the trained strategy network is deployed on the local terminal of the micro-grid, and combined with the joint market transaction mechanism, the online action generation, market bidding, regulation and control execution and external interaction feedback of the agent are realized, thereby building a distributed energy-carbon collaborative regulation and control system with closed-loop optimization capability, and improving the overall operation efficiency and low-carbon level of the system. BRIEF DESCRIPTION OF DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the present application, the drawings needed in the description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0060] Figure 1 The step flow chart of the micro-grid electricity-carbon combined transaction method in the embodiment;

[0061] Figure 2 The physical architecture diagram of the regional interconnected micro-grid in the embodiment;

[0062] Figure 3 The local electricity-carbon combined market transaction flow chart in the embodiment;

[0063] Figure 4 The network architecture diagram of MAPPO-TDSA in the embodiment;

[0064] Figure 5 The algorithm flow chart of MAPPO-TDSA in the embodiment. DETAILED DESCRIPTION

[0065] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described, obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0066] Please refer to Figures 1-4 As shown in the drawings, the present application provides a micro-grid electricity and carbon combined transaction method, comprising the following steps:

[0067] Step S1, a regional interconnected micro-grid system model is constructed, each micro-grid is modeled as an intelligent agent with local load, renewable energy power generation equipment, energy storage carbon fixation device and controllable gas turbine, and power output, carbon emission and operation cost models of each equipment are defined;

[0068] Step S1-1, define the local load, energy storage carbon fixation device, controllable gas turbine and renewable energy power generation equipment contained in each micro-grid, and establish the power output model and carbon emission model of each equipment;

[0069] Step S1-2, the energy storage carbon fixation device adopts an integrated battery structure with energy storage and carbon dioxide absorption functions, and sets state of charge, charging and discharging efficiency, carbon fixation efficiency and maximum carbon fixation power constraints;

[0070] Step S1-3, the output power and operation cost of the controllable gas turbine form a nonlinear quadratic function relationship, the carbon emission is calculated according to the fuel carbon content factor, thermal efficiency and combustion oxidation rate, and there are maximum power, minimum power and climbing rate constraints;

[0071] Step S1-4, the renewable energy equipment includes a wind power generation module and a photovoltaic power generation module;

[0072] The wind power generation module adopts an interval prediction modeling method, and the upper and lower limit output prediction interval is constructed based on historical wind speed data and meteorological factors;

[0073] The photovoltaic power generation module adopts a point prediction method, and a single output value is calculated using the sunshine intensity and radiation model;

[0074] In some embodiments, the infrastructure of each micro-grid includes user load, carbon fixation energy storage battery, controllable distributed gas turbine, and renewable energy generator, each device is equipped with corresponding power sensor and carbon flow detection and accounting device. The integration of these components enables the micro-grid to effectively manage power supply and carbon emissions, thereby achieving more efficient energy use and environmental benefits.

[0075] Local user load:

[0076] The electricity demand of residents has strong periodicity in time domain, and the peak and valley values of power demand usually fluctuate in units of days. Therefore, by collecting and analyzing historical load data and related influencing factors, the power demand prediction value of microgrid i at time t can be obtained using machine learning algorithms such as deep forest and Bayesian optimization , while the real demand value of microgrid i will be observed after local electricity-carbon joint market transactions.

[0077] Energy storage and carbon sequestration battery:

[0078] The rapid development of carbon capture, utilization and storage technology and research on lithium-dioxide batteries provide a theoretical basis for the development of carbon sequestration and energy storage integrated devices. The latest research results confirm the feasibility of developing new battery systems with dual working models of energy storage and carbon sequestration. Such devices can serve as both battery energy storage and carbon dioxide absorption devices, using electrical energy to fix carbon dioxide, providing greater flexibility for microgrid energy scheduling.

[0079] The energy storage and carbon sequestration battery model constructed in this embodiment is:

[0080]

[0081]

[0082] wherein, SoC represents the state of charge of the energy storage and carbon sequestration battery at time t; the maximum capacity of the energy storage and carbon sequestration battery is , and the battery capacity needs to be selected flexibly according to the actual battery usage to reduce infrastructure costs; the battery decay rate represents the degree of power loss during battery operation; and represent the charging and discharging efficiency of the battery, respectively; the power values of the battery operating in energy storage mode and carbon sequestration mode are represented as and , respectively; the indicator functions and are used to adjust the working state of the carbon sequestration and energy storage battery. For the indicator function , when the parameter , the value is 1, otherwise the value is 0; for the indicator function , when the parameter , the value is 0, indicating that it is working in carbon sequestration mode, otherwise the value is 1, indicating that it is working in energy storage mode; the carbon sequestration amount of the energy storage and carbon sequestration battery at time t is , and the carbon sequestration efficiency is .

[0083] To ensure the safe, stable and effective operation of the energy storage and carbon sequestration battery, the following constraints need to be met:

[0084] The state of charge must be operated within the range allowed by design to avoid overcharging or over-discharging.

[0085]

[0086] wherein, is the state of charge at the current time; , are the lower and upper limits of the battery state of charge, respectively.

[0087] The charge and discharge power must be limited within the rated power range to protect the battery life and system safety.

[0088]

[0089] wherein, is the charge and discharge power at the current time (negative for discharging, positive for charging); and are the maximum discharge power and maximum charge power, respectively.

[0090] The carbon sequestration power must be limited within the design capacity of its carbon sequestration device to ensure carbon sequestration efficiency and safe operation.

[0091]

[0092] wherein, is the carbon sequestration power at the current time; is the maximum carbon sequestration power.

[0093] If the energy storage carbon sequestration battery is working in carbon sequestration mode, it can use electrical energy to absorb carbon dioxide gas and convert it into solid carbon, and after approval, it can obtain carbon quota is:

[0094]

[0095] wherein, is the emission factor of the regional interconnected microgrid group's local power grid; is the carbon sequestration amount of the energy storage carbon sequestration battery at time t.

[0096] Controllable distributed gas turbine:

[0097] Renewable energy generation has strong randomness and large fluctuations, and there is a generation window period, such as no wind or weak light at night, so in this embodiment, the microgrid under study is equipped with a controllable conventional generator set composed of multiple gas turbines to ensure the power demand of users under any conditions. The gas turbine uses natural gas to produce water vapor, which in turn drives the generator to generate thermal power. A large amount of carbon dioxide is emitted during the power generation process, and the carbon sequestration amount of the energy storage carbon sequestration battery at time The carbon emissions generated are:

[0098]

[0099]

[0100] Among them, the carbon dioxide emissions produced by the gas turbine unit at time t are ; This represents the output power of the controllable gas turbine unit equipped in microgrid i at time t; the carbon emission coefficient of the unit is... Its calculation is affected by the carbon content factor of the fuel. (Refers to the amount of carbon contained in a unit volume of fuel gas), oxidation coefficient (This coefficient is used to correct the calculation, taking into account the impact of incomplete combustion and other factors on carbon emissions during actual operation, so that the results are closer to actual emissions.) Thermal efficiency of gas turbine units. (The proportion of input gas energy that is actually effectively utilized and converted into heat energy) and fuel calorific value The influence of four factors (referring to the heat released when a unit volume of gas is completely burned).

[0101] The operating cost of a gas turbine is typically quadratic in relation to its output power, reflecting the nonlinear characteristic of operating efficiency as power changes. The cost function is:

[0102]

[0103] in, It is the operating cost of the controllable gas turbine unit equipped in microgrid i over time t; , , It is the operating cost coefficient of the unit, when hour, This represents the fuel consumption cost of keeping the unit in standby mode.

[0104] When a gas turbine operates at low load, its combustion efficiency decreases significantly, potentially leading to unstable combustion or flameout. Incomplete combustion also results in a significant increase in carbon emissions, failing to meet environmental protection requirements. Furthermore, due to design capacity and material strength limitations, the unit has a maximum power limit. Exceeding this limit can cause overheating, excessive mechanical stress, or damage, significantly reducing equipment lifespan and increasing maintenance costs. The power output of a gas turbine cannot change instantaneously, as it is constrained by its mechanical and thermal inertia. Therefore, controllable gas turbine units require consideration of operational constraints during operation, including maximum and minimum power constraints and ramp-up constraints.

[0105]

[0106]

[0107] wherein the rated maximum output power and the minimum stable output power of the controllable gas turbine unit equipped in the microgrid i are denoted as and and the maximum ramping constraint is .

[0108] Renewable energy generator:

[0109] In the present embodiment, the microgrid under study is equipped with small wind turbines or solar photovoltaic panels, which generate green electricity (hereinafter referred to as green electricity) using wind energy and solar energy, and at the same time can obtain Chinese Certified Emission Reduction (hereinafter referred to as CCER) for trading in the carbon market to offset the payment of carbon emission quota. Since the renewable energy generator does not need external energy input during operation, only a small amount of operation and maintenance cost is required, so the generation cost is negligible.

[0110]

[0111]

[0112] wherein, is the approved carbon quota that the microgrid i generates green electricity and can trade in the carbon market; is the emission factor of the regional interconnected microgrid group in the area power grid; is the total amount of green electricity generated by the microgrid i at time t, and represent the output of the small wind turbine and the solar photovoltaic panel equipped in the microgrid i at time t, respectively.

[0113] In order to ensure the safe and stable operation of the renewable energy equipment, the equipment output is constrained within the maximum output power:

[0114]

[0115]

[0116] wherein, and represent the maximum output power of the small wind turbine and the solar photovoltaic panel equipped in the microgrid i, respectively.

[0117] Considering the actual situation of operation, consistent with the local load demand, at the beginning of each time t, the microgrid operation management agent cannot observe the actual output of renewable energy, which needs to be observed after the local electricity-carbon joint market energy transaction, and the agent can only rely on past data to predict the output of renewable energy equipment through machine learning methods to obtain the predicted value.

[0118] The wind speed is affected by many factors such as terrain, temperature gradient, and pressure system, and has significant instantaneous variation characteristics (such as turbulence). Although the solar radiation is affected by the cloud layer, the diurnal variation follows a relatively stable astronomical law, and there is no mechanical inertia effect. The volatility and complexity of wind speed are much higher than those of solar radiation, so wind power generation is more difficult to accurately predict than photovoltaic power generation. Therefore, wind power prediction needs to use interval prediction (such as probability distribution or confidence interval) to quantify uncertainty, and obtain the upper limit value and the lower limit value of the wind power prediction interval; the output curve of photovoltaic power generation has a strong linear relationship with irradiance, and the diurnal fluctuation mainly presents a predictable smooth transition with a low probability of extreme jump, so point prediction (single numerical prediction) is adopted to obtain the predicted value of the photovoltaic power generation.

[0119] Step S2, constructing a local electricity-carbon joint market mechanism, binding electricity flow and carbon emission flow in the electricity-carbon joint market, matching buy and sell orders in multiple rounds of bilateral auction, and determining the transaction result according to the joint bidding rule of electricity price and carbon price, and the transaction settlement is verified by the power flow constraint of the distribution line;

[0120] Step S2-1, establishing a local electricity-carbon joint market platform, regarding the microgrid as a market participant, and the market transaction object includes green electricity and corresponding carbon quota;

[0121] Step S2-2, setting a joint bidding rule, each microgrid as a buyer or seller submits an electricity price and a carbon quota price, and the price is dynamically generated by combining the predicted green electricity supply and load demand;

[0122] Step S2-3, using a multiple-round bilateral auction mechanism to match orders, arranging the buy-side price in descending order and the sell-side price in ascending order, and comparing the highest buy price with the lowest sell price in turn, and if the condition is met, the matching is achieved.

[0123] Step S2-3-1, constructing a buy-side order pool and a sell-side order pool according to the expected order set, and sorting them according to the price, arranging the buy-side price in descending order and the sell-side price in ascending order;

[0124] Step S2-3-2, performing round-by-round order matching operation on the ordered order pool, comparing the current highest bid price with the lowest ask price in each round, if the bid price is not lower than the ask price, determining as a tradeable order pair, and the matched transaction capacity is the minimum remaining capacity of both the buyer and the seller;

[0125] Step S2-3-3, after each round of order matching, performing power flow safety check based on the physical topology structure and line transmission capacity of the microgrid interconnection, and calculating whether the actual power transmission of each distribution line meets the capacity limit;

[0126] Step S2-3-4, if a certain round of matching transaction pair causes line overload, terminating the transaction settlement of the matching pair, and updating the order pool and entering the next round of matching;

[0127] Step S2-4, in each round of transaction, the settlement price is calculated by using the weighted average method according to the matching amount and the market supply-demand ratio, and the final transaction settlement amount is determined by the electricity price and the carbon price;

[0128] At the beginning of each time t, the management and operation agent of the microgrid i will determine the expected transaction volume and the expected transaction price of the microgrid i participating in the local electricity-carbon joint market according to the current state of the microgrid, including the state of charge of the local energy storage system , the predicted power generation of renewable energy (the upper limit value of the wind power prediction interval and the lower limit value ; the predicted value of photovoltaic power generation ) and the predicted demand value of local users .

[0129] Subsequently, these expected transaction volume and expected transaction price will be uploaded to the local electricity-carbon joint market management platform, and the settlement price, electricity transaction volume and the number of involved sellers and buyers will be determined through multiple rounds of bilateral auction mechanism. Multiple rounds of bilateral auction is a method of buying and selling through public competition to determine the price, and the specific process is as follows:

[0130] Market participants:

[0131] The microgrids participating in the local electricity-carbon joint market will be divided into buyers and sellers according to certain rules, and the expected transaction prices of the buyers and sellers are represented as and . Specifically, at time t, represents that the microgrid i wants to be a buyer, the transaction price is represented as , and wants to buy green power from the local market, ; represents that the microgrid i wants to be a seller, the transaction price is represented as , who wishes to sell green power to other microgrids , .

[0132] The order matching rules are as follows:

[0133] Order pool: In order to match orders, first, we need to sort the buy and sell orders by price according to the following rules: buy orders are sorted by price from high to low (buyers willing to pay more price are given priority), and sell orders are sorted by price from low to high (sellers willing to accept lower prices are given priority).

[0134] Let denote the buyer microgrid i, and the set of buyer orders is denoted as , which is sorted in descending order by . Assuming the number of buyers is m, the highest bid pair is denoted as , and the lowest bid pair is denoted as ; let denote the seller microgrid, and the set of seller orders is denoted as , which is sorted in ascending order by . Assuming the number of sellers is n, the lowest bid pair is denoted as , and the highest bid pair is denoted as . Therefore, the sorted order sets of buyers and sellers can be further represented as:

[0135]

[0136]

[0137] The total purchase amount and the total sale amount of the local electricity-carbon combined market at time t are:

[0138]

[0139] Order matching: After sorting the buy and sell orders, the system will perform multiple rounds of matching, each time comparing the highest bid and the lowest bid. When the buyer's bid is higher than or equal to the seller's bid, i.e. , the transaction is completed. After each round of matching is completed, the remaining amount of the buyer and seller in the order pool is updated, and then the matching continues until all eligible transactions are completed.

[0140] The highest bid and the lowest bid that match in the kth round are denoted as and , respectively. The corresponding expected transaction amount is also denoted as and . Among them, , , , .

[0141] For orders that satisfy , the transaction volume will confirm the available volume:

[0142]

[0143] where, is the transaction volume that can be achieved in this round, refers to taking the minimum value of a and b.

[0144] After the order achieves a transaction, the remaining volume of both buyers and sellers and the order pool are updated:

[0145] Buyer's remaining volume ;

[0146] Seller's remaining volume ;

[0147] The update rule for the buyer's order pool is expressed using the following formula:

[0148]

[0149] where the symbol is the set relative difference set operation, which means removing element from set ; the symbol is the union operation, which means adding the updated element to set . In the program, only the first element of the sorted list needs to be modified using a pointer.

[0150] The update rule for the seller's order pool is expressed using the following formula:

[0151]

[0152] where the symbol is the set relative difference set operation, which means removing element from set ; the symbol is the union operation, which means adding the updated element to set . In the program, only the first element of the sorted list needs to be modified using a pointer.

[0153] ​The settlement price is determined by the weighted value of the bid and ask prices of the trading parties that meet the transaction conditions. Compared with the arithmetic average pricing method, the weighted dynamic pricing method can better reflect the market supply and demand elasticity, reduce the risk of price manipulation, and improve market efficiency. The settlement price of the buyer and seller is represented as:

[0154]

[0155] where, , are the highest bid price and the lowest ask price of the buyer and seller in this round of matching orders, is determined by the market supply and demand ratio at time t:

[0156]

[0157] The transaction volume and settlement price of each round of matching are recorded in the local electric-carbon joint market management platform for subsequent settlement and statistics, and the final transaction volume and transaction amount are returned to the corresponding microgrids of the buyers and sellers. The collection of the local electric-carbon joint market at time t is , and each transaction is recorded in the following form:

[0158]

[0159] where the kth transaction records the transaction parties (i.e., the buyer microgrid i and the seller microgrid j) and their transaction volume and the settlement price .

[0160] Considering the line capacity constraints of the physical connection between microgrids, based on the DC power flow constraints, the power flow safety check is added in the order settlement stage, and the transaction combination that violates the network constraints is corrected.

[0161]

[0162] where, represents the set of n microgrids connected by power lines, is the set of l distribution lines. and are the specific operating point and current limit of the line. represents the amount of electricity that microgrid i needs to participate in transactions through the line, which is the transaction volume reached by microgrid i and other microgrids . represents the participation factor of microgrid i in line .

[0163] After considering the line constraints, the cumulative transaction volume of the buyer microgrid i and the seller microgrid j at time t is and is:

[0164]

[0165] Expenditure of the buyer microgrid i and the seller microgrid j in the local electricity-carbon joint market at time t and revenue is:

[0166]

[0167] Electricity trading volume of the microgrid i in the local electricity-carbon joint market at time t and transaction amount is:

[0168]

[0169]

[0170] wherein, if , it indicates that the agent acts as a buyer and spends to buy green electricity from the local market; if , it indicates that the agent acts as a seller and sells green electricity in the local electricity-carbon joint market, with a revenue of .

[0171] Average settlement price of electricity trading of the microgrid i in the local electricity-carbon joint market at time t is:

[0172]

[0173] The electricity-carbon joint market transaction involves two-way binding of electricity and carbon flow, and carbon quota is transferred with electricity trading. Carbon quota transferred by the microgrid i in the local electricity-carbon joint market transaction at time t is:

[0174]

[0175] wherein, is the emission factor of the regional interconnected microgrid group area power grid, is the electricity trading volume of the microgrid i in the local electricity-carbon joint market at time t.

[0176] Due to energy loss in the transmission process, the actual electricity volume traded by the microgrid i in the local electricity-carbon joint market is:

[0177]

[0178] In summary, after the local electricity-carbon combined market transaction is completed, the data returned to each microgrid management and operation agent includes the average settlement price , the actual transaction volume of electricity , the transferred carbon quota , and the transaction amount .

[0179] After the microgrid management and operation agent completes the local market energy transaction and energy management, at the end of time t, the actual output value of renewable energy and the load demand value can be observed. Based on these data, the agent evaluates the power supply and demand situation within the microgrid and decides whether to purchase or sell electricity from the external power market according to the power surplus or deficit, to ensure the balance of supply and demand in the microgrid. The balance of power supply and demand is as follows:

[0180]

[0181] The microgrid is connected to the external main grid through a public coupling point and transacts with the external power market based on real-time shared electricity price information. Assuming that the energy transacted by the microgrid i with the external power market at time t is , if , it indicates that the microgrid i needs to purchase electricity from the external power market to meet the local user demand, otherwise, if , it indicates that there is a power surplus and electricity needs to be sold to the external power market. The transaction amount of the two is:

[0182]

[0183] wherein represents the feed-in tariff, which is determined by the external power market, and the microgrid management operator sells electricity to the main grid through the external power market at the metering price at the point where the microgrid accesses the main grid; represents the wholesale price, which is determined by the external power market, and the microgrid management operator purchases electricity from the main grid through the external power market at the metering price at the point where the microgrid accesses the main grid.

[0184] When the microgrid is in a situation of power shortage, it needs to purchase electricity from the external power market. Since the main source of electricity in the external power market is thermal power generation, the purchase of electricity will result in indirect carbon emissions:

[0185]

[0186] wherein, represents the indirect carbon emission from the purchase of external power, is the emission factor of the regional grid where the regional interconnected microgrid cluster locates, is the amount of electricity purchased by microgrid i from the external power market at time t.

[0187] Microgrids are not included in the carbon quota allocation range because of their low carbon intensity, which is lower than the industry benchmark value, as they mainly use renewable energy to generate electricity. The initial carbon quota is 0, but it can be rewarded by approving its voluntary emission reduction (including green power and carbon sequestration) to offset the carbon emissions of other sources or traded in the carbon trading market.

[0188] Within the microgrid, the management of carbon emissions and carbon quotas involves multiple subjects: gas turbines produce direct carbon emissions by consuming traditional fossil fuels for power generation, and need to consume corresponding carbon quotas; purchasing electricity from the external power market produces indirect carbon emissions, which also need to consume carbon quotas. In contrast, energy storage carbon sequestration batteries can absorb carbon dioxide and convert it into solid carbon by using excess electricity, and can obtain carbon quotas after approval; at the same time, renewable energy generators produce green power using clean energy such as wind and solar energy, and can also obtain carbon quotas after approval. These subjects together form the carbon quota flow system within the microgrid. The net carbon quota of microgrid i at time t is:

[0189]

[0190] At the end of each energy management and dispatching cycle, i.e. , the microgrid management and operation agent will settle with the local residual carbon quota and the external carbon trading market.

[0191]

[0192]

[0193] where, is the residual carbon quota of microgrid i, and are the buy and sell prices of carbon quotas, respectively.

[0194] It is assumed that the microgrid management and operation agent trades in the external power market and the carbon trading market at an uneconomic price, which means that the agent will have more benefits by participating in the local electricity-carbon combined market.

[0195]

[0196] .

[0197] ​Step S3, based on the micro-grid model and the electricity-carbon market mechanism, a multi-agent Markov decision process model is constructed, and the state space, action space and multi-objective reward function are set;

[0198] Step S3-1, the operation decision process of each micro-grid agent is modeled as a Markov decision process, and the state transition of the Markov decision process depends on the observation state, action decision and environmental feedback at the current time;

[0199] Step S3-2, define the state space, action space and reward function triplets for each micro-grid agent, wherein the state space is composed of local observable variables, the action space is a continuous variable set, and the reward function is designed in combination with economic, environmental and constraint factors;

[0200] The state space includes: the upper limit value and the lower limit value of the predicted output of wind power generation, the predicted value of photovoltaic power generation, the predicted value of local user load, the state of charge of the energy storage carbon fixation device, the market settlement price and the carbon quota state of the last time step;

[0201] The action space is a continuous variable set, including: the expected output power of the controllable gas turbine, the expected charge and discharge power and the carbon fixation mode control parameter of the energy storage carbon fixation device, the expected offer and the expected traded electricity of the local electricity-carbon joint market;

[0202] The state space includes the upper limit value and the lower limit value of the predicted interval of wind power generation , the predicted value of photovoltaic power generation , the predicted value of residential load , the state of charge of the energy storage carbon fixation battery at the last time , and the average settlement price of the local electricity-carbon joint market .

[0203]

[0204] In the action space, the micro-grid contains two energy trading and management actions in the operation process: expected action of autonomous decision and actual action of interaction with the environment, both of which are continuous action space.

[0205] The expected action is determined by the management and operation agent of the micro-grid i based on the learned energy management and trading strategy at the initial time of each time t, according to the observed current state , including the expected output power of the controllable distributed gas turbine , the expected working power of the energy storage carbon fixation battery , the expected transaction amount and the expected transaction price of the micro-grid i participating in the local electricity-carbon joint market.

[0206]

[0207] Actual actions are generated by the output hard constraint verification and the multi-round bilateral auction mechanism of the local electricity-carbon combined market, including the actual output power of the gas turbine , the actual working power of the energy storage and carbon sequestration battery , the actual electricity trading volume participating in the local electricity-carbon combined market and the average settlement price .

[0208]

[0209] The specific verification method of the output hard constraint is as follows: after the agent gives the expected energy management and transaction, a conditional constraint verification is first performed. On the one hand, it is verified whether the output power of the gas turbine meets the power constraint and whether the power value is greater than or equal to zero when the energy storage and carbon sequestration battery works in the carbon sequestration mode, whether it meets the power constraint.

[0210]

[0211]

[0212] On the other hand, it is verified whether the expected trading volume is lower than the lower limit of green power generation prediction, so as to guarantee that the transaction target is green power generated by renewable energy generators as much as possible.

[0213]

[0214] Subsequently, the expected trading volume and the expected transaction price after the conditional constraint verification correction will undergo a bilateral auction mechanism to generate actual transaction actions, and return the transaction results to the agent, including the average settlement price , the actual electricity trading volume , the transferred carbon quota and the transaction amount .

[0215] The reward function includes an economic reward item, an environmental reward item, a false transaction penalty item and a dispatch out-of-bound penalty item;

[0216] The economic reward item is calculated by the transaction revenue minus the operating cost, the environmental reward item gives positive incentive according to the carbon sequestration amount and negative punishment according to the carbon emission amount;

[0217] The reward function is taken as the optimization target of the agent strategy network, and the state-action sequence generated by trajectory sampling is combined to update the backward gradient propagation of the strategy network;

[0218] The operation of microgrid involves multiple objectives, including maximizing economic benefits, minimizing carbon emissions, and rewarding functions are the core mechanism to drive agents to learn optimal strategies under the premise of meeting operational constraints and system stability. By quantifying these objectives into computable indicators, agents are guided to gradually achieve multi-objective optimization in complex environments.

[0219] Specifically, it includes economic rewards, environmental rewards, false transaction penalties, and operation overrun penalties.

[0220] Economic rewards:

[0221] The operating income of microgrid includes the revenue from selling green electricity to the local electricity-carbon joint market, the revenue from selling electricity to the external electricity market when there is power surplus, and the revenue from selling carbon quotas in the external carbon trading market after the dispatching period. The expenses include the cost of purchasing green electricity from the local electricity-carbon joint market, the cost of gas turbine output, the cost of purchasing electricity from the external electricity market when there is power shortage, and the cost of purchasing carbon quotas in the external carbon trading market after the dispatching period. The economic reward is the gross profit of microgrid operation :

[0222]

[0223] Environmental rewards:

[0224] One of the goals of microgrid operation is to achieve low-carbon operation through green energy generation, carbon capture technology, and carbon quota trading, pursue zero-carbon operation, and strive for "negative carbon operation" as much as possible. Under the premise of ensuring local user electricity demand, encourage the use of redundant electricity to absorb and fix carbon dioxide, encourage the use of green electricity through the local electricity-carbon joint market, and discourage the use of gas turbines for energy supply. Environmental rewards are rewards and penalties for carbon emission reduction and carbon emission behaviors in microgrid operation.

[0225]

[0226] wherein, is the reward coefficient for carbon sequestration behavior , is the penalty coefficient for carbon emission behavior , is the total power flow behavior within the microgrid, .

[0227] False transaction penalties:

[0228] Actual transaction volume greater than observed green electricity means that the microgrid does not sell all green electricity in the local electricity-carbon joint market, and there will be a false transaction penalty This penalty comes from the microgrid's dishonest operation and the false reporting of its green electricity trading volume, which is also consistent with the reality of data falsification in market operation.

[0229]

[0230] wherein, is the false operation penalty coefficient.

[0231] Operation overrun penalty:

[0232] The hard constraint check of action output ensures the safe and stable operation of the microgrid, but in the process of learning energy trading and management strategies, the decision actions output by the agent are inevitably beyond the operation limits, and the expected output power of the gas turbine and the expected working power of the energy storage carbon fixation battery overrun penalty will be set to constrain the safety boundary of the action space, and convert it into a learnable gradient signal to help the agent understand and learn the strategy that meets the operation constraint conditions.

[0233]

[0234]

[0235]

[0236]

[0237] wherein, , and are the overrun penalty terms of the gas turbine, battery state of charge and working power, respectively; , and are the overrun penalty coefficients of the gas turbine, battery state of charge and working power, respectively.

[0238] In summary, the overall reward function of the microgrid i at time t is:

[0239]

[0240] Therefore, the multi-objective optimal scheduling problem of the regional interconnected microgrid group can be re-expressed as:

[0241]

[0242] wherein, denotes the scheduling period, denotes the number of regional interconnected microgrids, and the microgrids solve the above problem by mutual cooperation.

[0243] Step S3-3, using a discrete-time model with a time step of 24, dividing the daily operation cycle into 24 time steps, and updating the reinforcement learning strategy at each time step to obtain an optimal control strategy based on dynamic markets and operating environments;

[0244] Step S3-4, based on the round experience sampling mechanism in reinforcement learning, collecting the state-action-reward-next state sequence data of the microgrid in the complete scheduling cycle as the sample basis for policy network training;

[0245] Step S4, using the multi-agent proximal policy optimization algorithm MAPPO-TDSA that integrates value decomposition and attention mechanisms to train the policy network of each microgrid agent, and output the optimal strategy for electricity-carbon joint market transactions and resource regulation;

[0246] The multi-agent proximal policy optimization algorithm MAPPO-TDSA is used to train the policy network, wherein each agent maintains a local policy network and critic network;

[0247] The optimization algorithm includes a task decomposition mechanism, which decomposes the overall reward function into multiple sub-task rewards, and establishes a sub-task value function for each sub-task;

[0248] The optimization algorithm includes an attention mechanism that dynamically adjusts the weight of each sub-task in the joint strategy evaluation;

[0249] The policy update is based on the information shared by neighboring nodes in the communication topology, and the distributed training process is realized through a consensus update rule;

[0250] For a multi-agent reinforcement learning problem involving K tasks, and there is no temporal relationship between tasks, the optimization problem can be represented as:

[0251]

[0252] wherein, , represents the objective function of all agents in the dth task, .

[0253] The overall reward function R can be decomposed into K sub-reward functions, i.e. ; The state value function and the Q value function (both follow the policy , are the parameters of the actor network / policy network ) can also be decomposed and represented as:

[0254] ;

[0255] ;

[0256] Advantage function Also can be expressed as:

[0257] .

[0258] For a Markov decision process with multiple tasks, let the optimal policy set be denoted as , is the optimal policy set for sub-task k, assuming an intersection between the optimal policy sets of sub-tasks.

[0259] The overall reward function for agent i is re-described as: .

[0260] where:

[0261] ;

[0262] ;

[0263] .

[0264] Energy transfer between microgrids within a region is interconnected, but agents rely on information flow for data transfer. The communication topology between agents is an undirected graph, denoted as , where the node set is , and the edge set is . Define the neighbor set as the set of neighbors of node i, i.e., node i can receive information from these neighbors. is the weight of the communication topology, if (including j = i), then , otherwise If , then the weight matrix is bidirectional random for all nodes i.

[0265] To handle continuous states and actions, a two-step actor-critic framework is adopted, which can operate on different time scales.

[0266] During the training process, the actor network receives the state and outputs the local policy . Subsequently, the critic network evaluates the value of the global policy by estimating , thereby updating the global state value function. By communicating with neighboring nodes, a consensus update method is used to estimate . The state value function is approximated by , where is a feature related to state s.

[0267] The update rule of the critic network is as follows: for a multi-agent reinforcement learning containing N agents, the policy gradient can be expressed as:

[0268]

[0269] wherein .

[0270] In order to avoid the use of global information, the scalar is introduced to estimate , which represents the optimization goal in reinforcement learning, that is, "maximize the expected cumulative reward".

[0271]

[0272] wherein represents the update step of the critic network, and the sub-reward function of agent i is . Therefore, under the dth task, the parameters of agent i are updated in the following way:

[0273]

[0274] wherein the local TD error is:

[0275]

[0276] Agent i updates the parameters of its actor network (policy network) in the following formula:

[0277]

[0278] wherein is the update step of the actor network.

[0279] As shown in Figure 5 , first, the hyperparameters and neural network parameters are initialized;

[0280] Then, it is judged whether the training iteration number is over. When it is judged that the training iteration number is over, the training is ended;

[0281] When it is judged that the training iteration number is not over, an empty set is initialized for storing experiences;

[0282] It is judged whether all agents output actions;

[0283] If there is an agent that does not output actions, agent i outputs the expected action according to the current state And re-evaluate whether all agents have outputted an action;

[0284] Once all agents have output actions, the actual action of agent i is obtained and executed through hard constraint testing and a bilateral auction mechanism in the local electricity-carbon joint market. We observed actual load demand, renewable energy generation, and trading prices in external electricity and carbon markets;

[0285] Determine whether the local training of all agents has ended. When the local training of all agents has ended, determine whether the consensus update of the judge network of all agents has ended.

[0286] Once the consensus update of the judge network of all agents is complete, the iteration counter is incremented and the process jumps to the step of determining whether the training iteration count has ended.

[0287] If the consensus update of the judge network of the agent has not ended, determine whether the training of the judge network of all sub-tasks has been completed.

[0288] Once the judge network for all subtasks has been trained, the consensus parameters are calculated and updated. :

[0289]

[0290]

[0291] Then, it is reassessed whether the consensus update of the judge network of all agents has ended;

[0292] If the agent's local training is not yet complete, compute and update:

[0293]

[0294] Update the parameters of the actor network:

[0295]

[0296] Through communication networks Send parameters Provide the argument node and re-evaluate whether the local training of all agents has ended;

[0297] If the training of the judge network for a subtask is not complete, calculate the reward value for that subtask. ,calculate And update;

[0298] Subsequently, the TD error was updated:

[0299] ;

[0300] updating parameters of the critic network

[0301]

[0302] rejudging whether the training of the critic network for all subtasks is completed.

[0303] Step S5, each microgrid generates expected transaction and scheduling actions according to the observed state using the trained strategy network, and after passing the hard constraint test, completes the matching and clearing of transaction orders based on the multi-round bilateral auction mechanism, determines the power transaction volume, carbon quota transfer and corresponding settlement price, and executes the resource scheduling behavior;

[0304] Each microgrid generates expected transaction actions according to the current observed state and performs hard constraint checking. The constraints include the upper limit of gas turbine power, the upper and lower limits of energy storage power, and the feasibility of carbon fixation power.

[0305] After passing the verification, the agent submits the transaction action to the electricity-carbon joint market and participates in multi-round bilateral auctions to determine the actual transaction pair, transaction volume, settlement price and carbon quota transfer.

[0306] The execution result is used as the input of the next round of state space and is used for training data expansion and model incremental optimization.

[0307] Step S6, according to the microgrid transaction execution result and the power carbon quota balance, energy compensation and carbon quota settlement are respectively carried out with the external power market and the carbon trading market, and the transaction feedback is used for the next round of strategy optimization.

[0308] The present application is to realize the joint optimization transaction and management of power flow and carbon emission rights in regional interconnected heterogeneous microgrids by combining deep reinforcement learning algorithm. Through real-time learning and adaptation of intelligent agents to dynamic changes under different operating environments, each microgrid can independently formulate and adjust power and carbon trading strategies, and through the electricity-carbon joint market, zero-carbon assets are converted into carbon quota income. Under the premise of policy exemption and mandatory quota allocation, the dual goals of "negative carbon operation" and external market arbitrage are realized, thereby improving the overall energy utilization efficiency, reducing the operating cost, and actively promoting the development of low-carbon economy. This model provides a standardized path for distributed energy to participate in the national carbon market under the new type of power system, effectively promoting the low-carbon transformation of energy structure.

[0309] In some embodiments, the present application provides a microgrid electricity-carbon joint transaction system, which comprises:

[0310] ​​The system modeling module is configured to establish an operation structure model of the regional interconnected microgrid. Specifically, each microgrid is modeled as an agent with local load, renewable energy generation units (including wind power and photovoltaic), energy storage and carbon fixation devices, and controllable gas turbines. In the model, the power output capacity, carbon emission calculation method, and operation cost function of each type of device are defined to support subsequent dispatch optimization and transaction decision-making.

[0311] The market mechanism construction module is configured to construct a local electricity-carbon joint market at the microgrid level. The market mechanism binds electricity flow and carbon emission flow, allowing each microgrid to submit joint bid orders based on its supply and demand and carbon quota status, including electricity price and carbon price. The market adopts a multi-round bilateral auction method for order matching. The transaction results are further combined with the physical topology and line capacity of the distribution network for power flow verification to ensure that all completed transactions are within the safety boundary of the power system.

[0312] The decision modeling module is configured to construct a Multi-Agent MDP model based on the above system model and market mechanism. The system defines a state space, action space, and reward function triple for each microgrid agent. The state space includes renewable energy prediction results, load prediction values, energy storage state, and historical market settlement data; the action space includes device control parameters and market transaction intentions; and the reward function is modeled based on economic benefits, carbon emission control targets, and behavior constraint penalties.

[0313] The strategy training module is configured to train the optimal joint transaction strategy of the microgrid agent. In this embodiment, a MAPPO-TDSA framework is constructed based on the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm and combined with the task decomposition mechanism and attention mechanism. Each agent maintains its local policy network and critic network, and the algorithm updates and optimizes the joint strategy through trajectory sampling, reward decomposition, and neighbor information sharing mechanism.

[0314] The strategy execution module is configured to output transaction and dispatch actions from the trained policy network based on the current observed state information during the actual operation of the system, and perform constraint verification on the output actions, including device capacity, adjustment rate, and carbon constraints. The verified actions are submitted to the local electricity-carbon joint market for multi-round bilateral auction to determine the transaction object, electricity quantity, carbon quota, and settlement price. The system then completes resource regulation and energy-carbon linkage execution based on the above results.

[0315] The external interaction module is used for compensatory interaction of energy or carbon assets with the external power market or carbon trading market after the local transaction is executed. The system automatically generates an order submitted to the external market according to the power balance and carbon quota of the microgrid, and inputs the interaction result as environmental feedback into the reinforcement learning model to realize incremental updating and adaptive evolution of the strategy.

[0316] In some embodiments, the present application provides a terminal, comprising:

[0317] a memory for storing a microgrid electricity-carbon combined transaction program;

[0318] a processor for executing the microgrid electricity-carbon combined transaction system to realize the steps of the microgrid electricity-carbon combined transaction method.

[0319] In some embodiments, the present application provides a computer readable storage medium, which stores computer instructions, and when a computer reads the computer instructions in the storage medium, the computer executes the microgrid electricity-carbon combined transaction method.

[0320] It can be understood that the systems, devices, modules or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, a laptop computer, a personal digital assistant, a tablet computer, a wearable device or a combination of any of these devices.

[0321] In a typical configuration, the computer includes one or more processors (CPU), input / output interface, network interface and memory.

[0322] The memory can include non-permanent memory in the computer readable medium, random access memory (RAM) and / or non-volatile memory such as read only memory (ROM) or flash memory (flash RAM). The memory is an example of the computer readable medium.

[0323] Computer-readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassettes, magnetic disk storage, quantum memory, graphene-based storage media, or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to computing devices. According to the definition in this embodiment, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0324] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0325] It should also be noted that the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, commodity or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, commodity or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, commodity or equipment including the element.

[0326] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve the desired result. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired result. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

[0327] The terminology used in this description is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments described in this description. As used in this description and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or," as used herein, refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0328] It will be understood that, although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a temporal sequence. Rather, these terms are used only as distinguishable to reference various information. For example, without departing from the scope of one or more embodiments described in this description, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the word "if' as used herein can be interpreted to mean "when" or "upon" or "in response to the determining."

[0329] The foregoing is merely illustrative of the principles of one or more embodiments described in this description and is not intended to limit the scope of one or more embodiments described in this description. The illustrative embodiments described in this description are intended to be only examples and are not intended to exhaust the scope of one or more embodiments described in this description. Any modification, equivalent replacement or improvement made without departing from the spirit and principle of one or more embodiments described in this description shall be included in the scope of one or more embodiments described in this description.

Claims

1. A micro-grid electricity-carbon combined transaction method, characterized in that, The method comprises the following steps: Step S1, constructing a regional interconnected microgrid system model, modeling each microgrid as an intelligent agent with local load, renewable energy generation equipment, energy storage carbon fixation device and controllable gas turbine, and defining the power output, carbon emission and operation cost model of each device; Step S2, constructing a local electricity-carbon combined market mechanism, binding electricity flow and carbon emission flow in the electricity-carbon combined market, matching buy and sell orders in multiple rounds of bilateral auction, and determining the transaction result according to the joint bidding rule of electricity price and carbon price, and the transaction settlement is verified by the power flow of the distribution line; Step S2-1, establishing a local electricity-carbon combined market platform, regarding the microgrid as a market participant, and the market transaction object includes green electricity and corresponding carbon quota; Step S2-2, setting a joint bidding rule, each microgrid as a buyer or seller submits an electricity price and a carbon quota price, and the price is dynamically generated in combination with the predicted green electricity supply and load demand; Step S2-3, order matching is performed by using a multiple-round bilateral auction mechanism, the buy price is arranged in descending order and the sell price is arranged in ascending order, and the highest buy price and the lowest sell price are compared in turn, and if the conditions are met, the matching is achieved; Step S2-3-1, constructing a buyer order pool and a seller order pool according to the expected order set, and sorting them according to the price in descending order and ascending order respectively; Step S2-3-2, performing round-by-round order matching operation on the sorted order pool, comparing the current highest buy price with the lowest sell price in each round, and if the buy price is not lower than the sell price, it is determined as a tradable order pair, and the traded electricity quantity is the minimum remaining quantity of the buyer and the seller; Step S2-3-3, after each round of order matching, based on the physical topology structure of the interconnected microgrid and the line transmission capacity, the power flow safety check is performed, and whether the actual power transmission of each distribution line meets the capacity limit is calculated; Step S2-3-4, if a round of matching transaction causes line overload, the transaction settlement of the matching pair is terminated, the order pool is updated, and the next round of matching is entered; Step S2-4, in each round of transaction, the settlement price is calculated by using the weighted average method according to the matching quantity and the market supply-demand ratio, and the final transaction settlement amount is determined by the electricity price and the carbon price; Step S3, based on the microgrid model and the electricity-carbon market mechanism, a multi-agent Markov decision process model is constructed, and the state space, action space and multi-objective reward function are set; Step S4, a multi-agent proximal policy optimization algorithm MAPPO-TDSA integrating value decomposition and attention mechanism is used to train the strategy network of each microgrid agent, and the optimal strategy for electricity-carbon combined market transaction and resource regulation is output; Step S5, each microgrid generates expected transaction and dispatching action by using the trained strategy network according to the observed state, and after passing the hard constraint test, the transaction order matching and clearing are completed based on the multiple-round bilateral auction mechanism, the electricity transaction quantity, carbon quota transfer and corresponding settlement price are determined, and the resource scheduling behavior is performed; Step S6, according to the micro-grid transaction execution result and the power carbon quota balance, energy compensation and carbon quota settlement are carried out with the external power market and the carbon trading market respectively, and the transaction feedback is used for the next round of strategy optimization.

2. The microgrid electricity-carbon co-trading method of claim 1, wherein, Step S1 includes the following steps: Step S1-1, define the local load, energy storage and carbon fixation device, controllable gas turbine and renewable energy power generation equipment contained in each micro-grid, and establish the power output model and carbon emission model of each device; Step S1-2, the energy storage and carbon fixation device adopts an integrated battery structure with energy storage and carbon dioxide absorption functions, and sets the state of charge, charging and discharging efficiency, carbon fixation efficiency and maximum carbon fixation power constraints; Step S1-3, the output power and operating cost of the controllable gas turbine form a nonlinear quadratic function relationship, the carbon emission is calculated according to the fuel carbon content factor, thermal efficiency and combustion oxidation rate, and there are maximum power, minimum power and climbing rate constraints; Step S1-4, the renewable energy equipment includes wind power generation module and photovoltaic power generation module; The wind power generation module adopts interval prediction modeling method, and the upper and lower limit output prediction interval is constructed based on historical wind speed data and meteorological factors; The photovoltaic power generation module adopts point prediction method, and a single output value is calculated by using sunshine intensity and radiation model.

3. The microgrid electricity-carbon co-trading method of claim 1, wherein, Step S3 includes the following steps: Step S3-1, model the operation decision process of each micro-grid agent as a Markov decision process, and the state transition of the Markov decision process depends on the observation state, action decision and environment feedback at the current time; Step S3-2, define the state space, action space and reward function triplets for each micro-grid agent, wherein the state space is composed of local observable variables, the action space is a set of continuous variables, and the reward function is designed in combination with economic, environmental and constraint factors; Step S3-3, a discrete time model with a time step of 24 is adopted, the daily operation cycle is divided into 24 time steps, and the reinforcement learning strategy is updated at each time step to obtain the optimal control strategy based on dynamic market and operating environment; Step S3-4, based on the round experience sampling mechanism in reinforcement learning, the state-action-reward-next state sequence data of the micro-grid in the complete scheduling period is collected as the sample basis for strategy network training.

4. The microgrid electricity-carbon co-trading method of claim 3, wherein, In step S3-2: The state space includes: the upper and lower limit values of the predicted output interval of wind power generation, the predicted point value of photovoltaic power generation, the predicted value of local user load, the state of charge of energy storage and carbon fixation device, the market settlement price and carbon quota state of the last time step; The action space is a set of continuous variables, including: the expected output power of the controllable gas turbine, the expected charging and discharging power and carbon fixation mode control parameters of the energy storage and carbon fixation device, and the expected quotation and expected transaction electricity quantity of the local electricity-carbon joint market; The reward function includes economic reward item, environmental reward item, false transaction penalty item and scheduling out-of-range penalty item; The economic reward item is calculated by subtracting the operating cost from the transaction revenue, and the environmental reward item gives positive incentive according to the carbon fixation amount and negative punishment according to the carbon emission amount; The reward function is taken as the optimization target of the agent policy network, and the state-action sequence generated by trajectory sampling is used to update the policy network through back propagation of the gradient.

5. The microgrid electricity-carbon co-trading method of claim 1, wherein, In step S4, the multi-agent proximal policy optimization algorithm MAPPO-TDSA is used to train the policy network, wherein each agent maintains a local policy network and critic network; The optimization algorithm includes a task decomposition mechanism, which decomposes the overall reward function into multiple sub-task rewards, and establishes a sub-task value function for each sub-task; The optimization algorithm includes an attention mechanism, which dynamically adjusts the weight of each sub-task in the joint policy evaluation; The policy update is based on the information shared by the neighbor nodes in the communication topology, and the distributed training process is realized through the consensus update rule.

6. The microgrid electricity-carbon co-trading method of claim 1, wherein, In step S5, each microgrid generates expected trading actions based on the current observed state and performs hard constraint verification, including gas turbine power upper limit, energy storage power upper and lower limits, and carbon sequestration power feasibility; After verification, the agent submits the trading action to the electricity-carbon joint market and participates in multiple rounds of bilateral auction to determine the actual trading pair, transaction electricity, settlement price, and carbon quota transfer; The execution result is used as the input of the next round of state space and is used for training data expansion and model incremental optimization.

Images