An electric-carbon coupling market user side autonomous decision method

Abstract

The application relates to the technical field of power systems, and particularly discloses a kind of electric carbon coupling market user side autonomous decision-making method, the method comprises the following steps: constructing an electric carbon market transaction model, the model is coordinated by carbon electricity coordinator, carbon electricity sensor is distributed to different distributed renewable energy DER, DER obtains a shared model from the carbon electricity market coordinator and carries out local training, then the local training model is combined to generate a global model; a partially observable Markov decision process with discrete time steps is established to manage the electric carbon market bid / ask of DER as a sequential decision process; based on the Markov decision process, each DER performs local training in the federal reinforcement learning model, and each DER derives its own electric carbon market transaction decision through a deep Q network. The application has the advantages that the proposed carbon electricity market framework can achieve a good trade-off between user-side transaction costs and the social total welfare of carbon electricity organizations.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power systems, and particularly relates to a user-side autonomous decision-making method for an electric-carbon coupled market. BACKGROUND

[0002] Excessive carbon emissions (CE) pose a serious threat to the global climate, such as severe weather events, rising sea levels, and destruction of ecosystems and biodiversity. In order to solve these problems, decarbonization is listed as a key priority by the power sector. With the substantial increase in the penetration rate of distributed energy resources (DERs) in distribution networks, carbon trading is considered as an important way to promote local decarbonization and renewable energy adaptation at the distribution level. How to fully encourage user-side participation in coupled electric-carbon trading at the distribution level and how to achieve the privacy and differentiation of distributed energy trading information are problems that need to be paid attention to. The existing technology realizes the exchange of energy and carbon quota between producers and microgrids based on a smart contract transaction platform, but ignores the limitations of the distribution network. The existing technology proposes a distributed photovoltaic electric-carbon trading operation mechanism based on cross-chain transaction technology, and the existing technology proposes a blockchain application for trading energy and carbon permits between interconnected microgrids. However, this power generation-based solution may impose higher CE costs on users, further leading to uneven incentives and weakening social welfare. The use side as the fundamental driving force for the supply side to produce consumer electronics should also bear internal economic responsibility. To this end, the existing technology proposes the concept of bilateral carbon trading to allocate CE costs to the user side and the supply side.

[0003] Although the CE obligation on the user side is widely recognized, it is still in its infancy for actual implementation due to the short-sighted consideration of CE allocation from the supply side of the power sector. And so far, there is little research in the field of carbon electricity market, and carbon electricity trading is limited to a small range of typical homogeneous end-users, which deviates from the development trend of the user-side carbon electricity market. Under the drive of demand-side electrification, more heterogeneous distribution networks from different industries, in different geographical locations, with different consumption patterns should be encouraged to participate in the electricity distribution market. The peer-to-peer (P2P) trading paradigm has become a viable coordination architecture, and existing technologies take into account the heterogeneous tendency of DER while achieving transaction privacy. P2P mode can be roughly divided into system-centered mechanism, user-centered mechanism and hybrid mechanism. In the system-centered mechanism, the central coordinator (usually the distribution network operator (DSO)) basically participates in collecting the electricity-carbon market offers of DER and optimizes the electricity-carbon market transactions of peers to maximize social welfare. This mechanism is heavily dependent on the central coordinator and can prove infeasible due to scalability issues and high operation and maintenance costs [7]. As an alternative, nodes in the user-centered mechanism make decisions and communicate directly without a central coordinator, where decentralized models such as primal-dual gradient, consensus algorithm, and alternating direction multiplier are often used. However, the lack of a central coordinator can lead to market efficiency and global social welfare not meeting expectations. In this context, hybrid designs have been further developed, such as supply-demand ratio in P2P trading, intermediate market interest rate and bill sharing. In this mechanism, trading nodes are forced to share the same clearing price, which contradicts the basic idea of P2P market, i.e., to obtain specific pricing signals for electricity-carbon market traders with self-interest.

[0004] In addition, in the above three mechanisms, the electricity-carbon market relies heavily on accurate parameters, reliable uncertainty estimates, and complete economic models, which are often unrealistic in reality. To solve this problem, reinforcement learning (RL) as the next generation of decision-making method can have great advantages in capturing system uncertainty and adapting to various state dynamics. However, the centralized learning algorithm has the disadvantages of scalability and privacy due to the forced acquisition of all agents' local observations and actions. To solve this problem, federated learning (FL) technology has begun to be integrated into reinforcement learning, forming federated reinforcement learning (FRL) algorithm. Specifically, the natural distribution characteristics of FRL can achieve sufficient local model training without sharing information. So far, FRL has been increasingly concerned in the application of electricity trading, such as integrated energy systems, building communities, microgrids, etc. It should be pointed out that from the existing technology, it can be seen that the research of FRL in electricity trading still lacks sufficient consideration of heterogeneous DER with differentiated objectives on the user side, which limits the advantages of FRL in making adaptive decisions for DER in a heterogeneous market environment. SUMMARY

[0005] The purpose of the present application is to overcome the shortcomings of the prior art and provide an electricity-carbon coupled market user-side autonomous decision-making method.

[0006] The purpose of the present application is achieved by the following technical solution: an electricity-carbon coupled market user-side autonomous decision-making method, the method comprising:

[0007] An electricity-carbon market transaction model is constructed, which coordinates carbon and electricity sensors through a carbon and electricity coordinator, distributes carbon and electricity data to different DERs, and obtains a shared model from the carbon and electricity market coordinator and performs local training, then merges the local training model to generate a global model; the DERs include a subway model, an aggregated electric vehicle model, a PV photovoltaic model, a carbon emission unit model, and a load aggregator model;

[0008] A partially observable Markov decision process with discrete time steps is established to manage the electricity-carbon market bidding / asking of DER as a sequential decision process;

[0009] Based on the Markov decision process, each DER performs local training in the federated reinforcement learning model, and each DER derives its own electricity-carbon market transaction decision through local self-training.

[0010] Specifically, the subway model is:

[0011] (1a)

[0012] (1b)

[0013] (1c)

[0014] (1d)

[0015] (1e)

[0016] (1f)

[0017] (1g)

[0018] (1h)

[0019] wherein formula (1a) represents the total number of running subway stations; formula (1b) represents the total running time from the starting station to the terminal station; formula (1c) limits the running time between each station; formula (1d) limits the operating ISOE of OESD of each station interval; formula (1e)-(1g) are to coordinate the running time and the electricity-carbon market trading period; is the unit electricity-carbon market trading period; and is the first station interval in the t th electricity trading period; is the running time covering the entire interval in the t th electricity trading period, calculated as the sum of the running times of the th interval; the first station interval in the t th trading period is also the last station interval in the t -1th trading period ; the last station interval in the t th trading period is the first station interval in the t +1th trading period ; the sum of and of two adjacent trading periods is equal to the running time of the station interval; formula (1h) represents the net power consumption of the subway during the t th electricity-carbon market trading period, calculated by the function :

[0020] (1i)

[0021] wherein and is pre-calibrated constant.

[0022] The aggregated electric vehicle model is:

[0023] (2a)

[0024] wherein formula (2a) represents the charging power of the aggregated EV is subjected to a pre-calibration interval by which the aggregated EV obtains the desired charging energy; the hyperparameter matrix H and J are used to describe the sequential charging space:

[0025] (2b)

[0026] wherein, is a matrix with all permutations rows containing and ; and are column vectors of length with values equal to 0 and 1, respectively, and, furthermore, and are the minimum / maximum charging power and each electric vehicle , furthermore, and are calculated by each individual electric vehicle minimum / maximum charging power , calculated as follows:

[0027] (2c)

[0028] (2d)

[0029] (2e);

[0030] wherein: is the upper bound of the aggregated electric vehicle charging power; is the lower bound of the aggregated electric vehicle charging power; is the upper bound of the charging power of the n th electric vehicle; is the lower bound of the charging power of the n th electric vehicle;

[0031] The PV photovoltaic model is:

[0032] (3a)

[0033] wherein:​ for t Real-time photovoltaic power generation; for t Lower limit of photovoltaic power generation at any given time; The upper limit; Power factor; Need to stay Within the range.

[0034] Specifically, the carbon emission unit model is as follows:

[0035] (4a)

[0036] (4b)

[0037] (4c)

[0038] (4d)

[0039] (6a)

[0040] (6b)

[0041] (7a)

[0042] (7b)

[0043] (7c)

[0044] (7d)

[0045] (8)

[0046] (9)

[0047] (10a)

[0048] (10b)

[0049] (10c)

[0050] (10d)

[0051] (11)

[0052] In the formula: Let g be the unit commitment variable for generator set g at time t (1 for on, 0 for off). For distributed generation g exist t Output at any moment; and For distributed generation g exist t The lower and upper bounds of the output at each moment; and for t Time generator set g The lower and upper bounds of the active ramp capacity. and Let be the minimum online and offline time of generator set g at time t. For the first j Carbon emissions from secondary carbon-containing materials; This refers to the carbon content in the carbon raw material.

[0053] Formula (4a) represents the active power generation limit; Formula (4b) represents the active power ramp-up limit; Formulas (4c)-(4d) represent the online time and minimum offline time constraints; For the first d Individual load aggregators t The basic load at any given time; and For the first d Individual load aggregators t Minimum and maximum load at any given time; and For the first d The maximum and minimum demand responses of the load aggregator at time t. Formula (5a) limits the demand response after the first... d The load of a load aggregator; Formula (5b) limits the range of demand response; Formula (5c) represents the load factor requirement. This is the direct carbon emission from cement plants. Indicates the carbon emission factor; Indicates the output during the combustion and clinker process; To consume power; , and It is a constant coefficient; Direct carbon emissions from aluminum electrolysis plants; Carbon emissions from fuel combustion Carbon emissions from raw material consumption; Carbon emissions from electrolytic aluminum production; f Indicates fuel index; This indicates the average lower heating value, net consumption, and carbon oxidation rate of renewable fuels; This indicates the carbon content per unit calorific value of renewable fuels; This indicates the power consumption of an electrolytic aluminum plant; and is a constant parameter for describing the raw material consumption in the production of electrolytic aluminum; and represents the constant CE coefficient; is the direct cost efficiency of the steel plant; represents the CE coefficient of coal, natural gas and other materials; represents the coal and gas required for steel production; represents the steel production and coal gas production; is the direct carbon emission of the oil refinery; represents the raw oil consumed by the oil refinery; represents the mass fraction of coke in the oil refining product; represents the utilization efficiency; represents the i energy and the corresponding CE coefficient; is the direct carbon emission of the petrochemical device; is the carbon emission of fossil fuel combustion, is the carbon emission of carbon-containing raw material consumption, is the carbon emission of calcium carbonate decomposition; represents the fossil fuel index, is a constant coefficient for describing the i fuel consumption, represents the index of carbon-containing raw materials; represents the consumption of carbon-containing raw materials; represents the production of carbide and purified ash; represents the carbon content in carbide and purified ash; represents the consumption of carbonates; represents the CE coefficient of carbonates and carbonate purity; is the direct carbon emission of the business center, A is the surface area of the business building; is the j CE coefficient of the carbon-containing material; is the k average distance of the transportation mode, the weight of the carbon-containing material and the carbon emission coefficient; is the h CE coefficient of the energy and the energy consumption of the demolition process; is the service life and the daily operating hours.

[0054] Specifically, it also includes constructing a load aggregator model, which is:

[0055] (5a)

[0056] (5b)

[0057] (5c)

[0058] Among them, formula (5a) restricts the demand response after the first d The load of a load aggregator; Formula (5b) limits the range of demand response; Formula (5c) represents the load factor requirement.

[0059] Specifically, the electricity carbon market trading model is as follows:

[0060] (12a)

[0061] (12b)

[0062] (12c)

[0063] (12d)

[0064] (12e)

[0065] (12f)

[0066] (12g)

[0067] (12h)

[0068] (12i)

[0069] (12j)

[0070] (12k)

[0071] (12l)

[0072] (12m)

[0073] (12n)

[0074] (12o)

[0075] (12p)

[0076] (12q)

[0077] In the formula: For nodesb the dielectric constant of the material; the conductance of node b the active power of node b at time t; the reactive power of node b at time t; the power flow of distribution line l the maximum power flow; and the power flow of distribution line l at time t ; the active / reactive power flow; and the power flow of distribution line l at time t ; the squared current along distribution line b at time t ; the squared node voltage magnitude at node b at time t ; the carbon emission intensity of node l at time t ; the carbon emission intensity of distribution line t accompanying the power loss along distribution line l at time ; the indirect carbon emission of node b at time t ; the direct carbon emission of node at time b ; the total carbon emission of node t at time ; the amount of carbon emission permits purchased from external carbon emission market; b t at time ; the amount of traded carbon permits of node at time b ; wherein, equations (12a)-(12b) represent the node active and reactive power balance; equations (12c)-(12d) constrain the bidirectional branch power flow; equation (12e) represents the branch voltage drop; equation (12f) represents the power flow by second order cone relaxation; equation (12g) represents the node voltage range; equation (12h) couples t and with power factor ; equation (12i) limits ​​​​​​The range; Formula (12j) indicates CE balance, where the right hand indicates the total CE from the CE unit; and the left hand divides the total CE into three parts: the CE of the power seller in Formula (12j-1), the CE of the power buyer in Formula (12j-2), and the CE associated with branch power loss in Formula (12j-3); Formula (12k) calculates the CE associated with branch power loss; Formula (12l) calculates the node CE strength by weighted average CE of all injected power; Formula (12m) divides the branch l Carbon strength is defined from the branch l Carbon intensity of the outflowing node; Formulas (12n)-(12o) calculate the indirect CE of the power seller and buyer; Formula (12p) adds the indirect CE and the direct CE to obtain the total CE; Formula (12q) indicates that the CE of bus B should always be below its carbon allowance.

[0078] Specifically, the portion of the discrete time step observable Markov decision process is achieved through... ,definition, N One agent, state set S Private Observation Set O Action Set A Return function set R State transition function T and discount factor gamma ;

[0079] State set Including partial observations of all agents at time t, the local observations are described as follows:

[0080] subway:

[0081] (13)

[0082] In the formula: For nodes b exist t Voltage amplitude at any given moment; For power distribution lines l exist t Carbon emission intensity at any given moment; The initial carbon permitting of node b at time t; For the operating characteristics of the power distribution network; For transaction information; For the subway's operating parameters, here, express Up / down time; Indicates time t From the To the ISOE settings between stations;

[0083] Other e-carbon participants:

[0084] (14)

[0085] Action set For all agents, the action at time t is defined as:

[0086] Subway:

[0087] (15)

[0088] In the formula: is the electricity trading volume of node b at time t; Node b The transaction carbon permit amount at time t ; , carbon permit trading volume , and operating time and ;

[0089] Other e-carbon participants:

[0090] (16)

[0091] State transition T: through the function , the environment state is associated with the action ;

[0092] Reward function set R: the individualized goal of DER calculated according to the obtained action, the reward function of all DERs has the same composition, which is represented by the following formula:

[0093] (17a)

[0094] In the formula: represents the carbon tax price, which is calculated by the product of carbon intensity and carbon tax ; represents the reward function of DER i, reflects the transaction utility of the individualized goal; is the carbon emission cost; is the network usage cost; is the bilateral carbon trading cost.

[0095] Specifically, the transaction utility is modeled as a quadratic function:

[0096] (17b)

[0097] where, for metro and aggregated electric vehicles: denotes energy saving, , and are positive constants, indicating and have a clear monotonic relationship;

[0098] for PV photovoltaic: denotes PV consumption, where and are negative constants;

[0099] for carbon emission units: denotes generation cost, where and are positive constants;

[0100] for load aggregators: denotes negative utility for indicating expected failure of power consumption, where and are positive constants;

[0101] bilateral carbon trading cost is modeled as a linear function of carbon permits:

[0102] (17c)

[0103] where: is the bilateral carbon trading cost; is t the carbon emission trading volume between i and m two carbon trading subjects at time

[0104] Specifically, the training process of the federated reinforcement learning model includes global aggregation of updated local models and actor-critic network used for local model training;

[0105] Let I denote a set of DERs, and denote a dataset, each DER trains its own local model and passes the model to the electricity-carbon market coordinator to determine the global model denoted as , and then broadcasts the global model to all DERs for local update, so as to realize the differentiated goals of heterogeneous DERs in a self-determination and privacy protection manner.

[0106] Specifically, WFTD3 is used as the reinforcement learning algorithm for each DER to train its local model in the electric carbon market. In WFTD3, each DER has an actor-commentator framework, consisting of one actor network and two commentator networks, with parameters... DER i Actor network It works in a decentralized manner, taking local observations As input, and to generate deterministic actions. Then pass it on to the criticism network. This outputs a Q-value estimate for action evaluation; the target actor network is then incorporated into WFTD 3. and Target Critics Network Its soft update formula is:

[0107] (18)

[0108] In the formula The target update factor is updated by the commenter network through temporal difference learning, while the actor network updates it through deterministic policy gradients.

[0109] (19)

[0110] (20)

[0111] The weighted average field technique is used to describe the neighbor agent's effect on the first... i The impact of individual DER behaviors:

[0112] (twenty one)

[0113] In the formula: For network parameters; For sample size; The loss function; for t The target Q value at time t; Discount factor; The state at time t; For actor networks in Actions generated under a given state; The Q-value predicted by the Critic network; The reward that an agent receives from the environment after performing an action; The optimization objective of the Actor network; The gradient with respect to the parameters; for t The state at any given moment; Indicates the first ia weighted action of the DER, a neighbor agent representing the i DER, a slight perturbation is represented, and then the Taylor theorem is used to derive the pairwise Q-value function

[0114] (22)

[0115] where, since the first term can be eliminated; the second term, as the remainder of the Taylor polynomial , can be obtained when is M-smooth.

[0116] In particular, meta-function and time-varying control variants are employed to reconcile the local models trained in WFTD 3 with the global model of the electricity carbon market reconciler , which includes meta-function and time-varying control variants;

[0117] The meta-function is formulated as:

[0118] (23)

[0119] where: w is the network parameter; β denotes the step size; is the gradient of the meta-function; I is the sample size; the meta-function is represented by the average of the meta-function , where the function of the i th DER is defined as:

[0120] (24)

[0121] To update the local model, the gradient of equation (20) is calculated , which is calculated as follows:

[0122] (25)

[0123] Further development of the unbiased estimate:

[0124] (26)

[0125] where: is the unbiased estimate; denotes the size of D i ; and is the evaluation model w based on the input data d ​Predicted true label L error at time t; by its unbiased estimate calculated by independent batches , and Further solve:

[0126] (27)

[0127] A random control variable is proposed for each DER Wherein the average control variable is initialized for the electricity-carbon market coordinator , using the obtained , the update of the local model can be defined as:

[0128] (28)

[0129] Then, the random control variable is also updated via the calculated gradient ,

[0130] (29)

[0131] Using formula (24) and formula (25), then aggregate the local updates to update the parameters of the electricity-carbon market coordinator,

[0132] (30)

[0133] Through formula (23)-(30), a round of communication and update between DER and electricity-carbon market coordinator is completed.

[0134] The present application has the following advantages:

[0135] 1. The present application proposes a new E&C market framework, which enables heterogeneous DERs to achieve their differentiated goals at the user side, and constructs an interactive operation model of typical urban rail transit, especially for the rarely developed subway rail transit.

[0136] 2. In order to perfect and adapt to the actual carbon market, the present application newly establishes a direct CE model of typical high-carbon enterprises, and incorporates it into the bilateral DERs transaction at the distribution level.

[0137] 3. The present application proposes a pFedScv framework combined with WFTD 3 algorithm to solve the carbon electricity transaction problem, which allows users to make autonomous decisions and privacy protection during the carbon electricity transaction process.

[0138] 4. The proposed carbon electricity market framework can achieve a good trade-off between user-side transaction cost and social total welfare of carbon electricity organization BRIEF DESCRIPTION OF DRAWINGS

[0139] Figure 1 The pFedScv-based user autonomous electric-carbon coupling market mechanism schematic diagram of the present application;

[0140] Figure 2 The subway operation time and electric service handling cycle cooperation schematic diagram of the present application;

[0141] Figure 3 The subway measurement point schematic diagram of the present application;

[0142] Figure 4 The pFedScv framework schematic diagram of the present application;

[0143] Figure 5 The real electric-carbon distribution market;

[0144] Figure 6 The individualized target cost schematic diagram of five representative DERs under three electric-carbon market frameworks;

[0145] Figure 7 The CE cost schematic diagram of five representative DERs under three electric-carbon market frameworks;

[0146] Figure 8 The network usage cost schematic diagram of five representative DERs under three electric-carbon market frameworks;

[0147] Figure 9 The bilateral carbon trading cost schematic diagram of five representative DERs under three electric-carbon market frameworks;

[0148] Figure 10 The first type of electric power transaction schematic diagram;

[0149] Figure 11 The second type of electric power transaction schematic diagram;

[0150] Figure 12 The third type of electric power transaction schematic diagram;

[0151] Figure 13 The first type of carbon emission permit transaction schematic diagram;

[0152] Figure 14 The second type of carbon emission permit transaction schematic diagram;

[0153] Figure 15 The third type of carbon emission permit transaction schematic diagram;

[0154] Figure 16 The electric power transaction result schematic diagram in case 1;

[0155] Figure 17 Power transaction result illustration for case 2;

[0156] Figure 18 Carbon transaction result illustration for case 1;

[0157] Figure 19 Carbon transaction result illustration for case 2;

[0158] Figure 20 Convergence curve and stability illustration for four FL algorithms for load aggregators;

[0159] Figure 21 Convergence curve and stability illustration for four FL algorithms for aggregated electric vehicles;

[0160] Figure 22 Convergence curve and stability illustration for four FL algorithms for distributed energy sources with CE;

[0161] Figure 23 Convergence curve and stability illustration for four FL algorithms for metro;

[0162] Figure 24 Convergence curve and stability illustration for four FL algorithms for photovoltaic;

[0163] Figure 25 Box plot for four FL algorithms. DETAILED DESCRIPTION

[0164] In order to make the objects, technical solutions, and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application, that is, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. The components of the embodiments of the present application generally described and shown in the drawings herein can be arranged and designed in various different configurations.

[0165] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0166] It is to be noted that the relational terms herein, such as first and second and the like, are used solely to distinguish one from another entity or action without necessarily requiring or implying any actual relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0167] The application will be further described below in connection with the drawings, but the scope of protection of the application is not limited to the following description.

[0168] As shown in the figure, an electricity-carbon coupling market user side autonomous decision-making method, the method comprises: Figures 1 to 25

[0169] An electricity-carbon market transaction model is constructed, the model coordinates carbon electricity sensors through a carbon electricity coordinator, distributes carbon electricity data to different DERs, the DERs obtain a shared model from the carbon electricity market coordinator and perform local training, and perform local training for individualized goals to adjust their local strategies, and then the carbon electricity market coordinator merges the local training models and generates a global model; the DERs include subway models, aggregated electric vehicle models, PV photovoltaic models, carbon emission unit models and load aggregators models; heterogeneous DERs with differentiated goals: in order to improve transaction efficiency and release transaction potential, the application considers the heterogeneity of five representative DERs with different geographical locations, transaction modes and transaction goals, in addition to the common minimization of carbon electricity market transaction cost, the differentiated goals of such DERs involve subway and aggregated EV: energy saving; PV: maximizing local consumption; CE unit: minimizing power generation cost; load aggregator: minimizing negative utility, all these differentiated goals are formulated as reward conditions for effective solutions, in order to more effectively manage the carbon electricity market, some small-scale users are aggregated to form load aggregators, the load aggregators are divided into five types according to geographical areas, including IZ, CZ, RZ, PZ and AZ; direct CE units include cement plants, electrolytic aluminum plants, steel plants, petrochemical plants, oil refineries and commercial centers, direct CE refers to CE generated at the user side due to a series of production and operation activities, such as burning, decomposition and consumption of fossil fuels, etc.;

[0170] ​Metro is a relatively underdeveloped field, but should be an important DER in the electricity-carbon market distribution market, from the perspective of electricity, it can obtain power from the traction system and inject power into the power distribution network through regenerative braking, from the perspective of carbon, metro is undoubtedly one of the most representative transportation departments, which contributes about 20% of CE among all users, for this reason, metro energy saving and decarbonization are considered important goals, traditional metro models mainly realize energy saving through operation, route management and regenerative energy utilization based on vehicle-mounted energy storage, without updating existing infrastructure, however, these models only discuss net electricity consumption from the perspective of a single section, rather than from the perspective of unit electricity, carbon market trading cycle, which makes it difficult for metro to participate in trading decisions, in order to overcome this challenge, the present application proposes a metro trading operation model, which coordinates between running time, multiple inter-station sections and unit electricity-carbon market trading cycle, specifically:

[0171] (1a)

[0172] (1b)

[0173] (1c)

[0174] (1d)

[0175] (1e)

[0176] (1f)

[0177] (1g)

[0178] (1h)

[0179] For ease of understanding, Figure 2 demonstrates the coordination between the running time of multiple inter-station sections and the electricity-carbon market trading time, wherein, u is the journey index of the metro line; k is the total number of metro stations within a one-way trip of the metro line; i is the inter-station section index; is the running time of the metro in the i section; is the running time of the metro in the iThe initial energy state of the segment; Formula (1a) represents the total number of subway stations in operation; Formula (1b) represents the total travel time from the first station to the last station; Formula (1c) limits the travel time between each station; Formula (1d) limits the operating ISOE of the OESD of the journey between each station; Formulas (1e)-(1g) are the coordination of travel time and the electricity carbon market trading cycle; The unit is the carbon market trading cycle; and For the first t The running time between the first and last stations within a single electrical dispatching cycle; For the first t The operating time covering the entire interval within a single electricity transaction cycle, according to the first... Calculate the sum of the running times for each interval; t The first inter-station interval of a trading cycle Also the first t -The last inter-station interval of a trading cycle ;No. t The last inter-station interval of a trading cycle It is the first t The first inter-station interval of +1 trading cycle Two adjacent trading periods and The sum equals The travel time between stations; the formula (1h) represents the travel time in the first station section. t The net electricity consumption of the metro during individual carbon market trading sessions is determined by the function calculate:

[0180] (1i)

[0181] In the formula, and These are pre-calibrated constant coefficients.

[0182] like Figure 3 As shown, the measurement point for the metro is the intermediate bus between the 110 / 35 kV converter and the 35 / 1.5 kV traction converter. In this way, the E&C transactions of the metro and other DERs can be attributed to the bus at the same voltage level, which is beneficial for calculation and analysis.

[0183] In practical applications, individual electric vehicles (EVs) are difficult to participate in the electricity market due to their limited capacity. To effectively manage large-scale EV trading, EV aggregation is necessary. Aggregated EVs act as participants in the electricity market through smart charging. Typically, the charging requirements of aggregated EVs are determined by six parameters: arrival and departure times. Upper and lower limits of charging rate and maximum and minimum charging energy These six parameters are sent from the electric vehicle driver to the charging station and are assumed to be consistent internally, on which basis, the operating cost constraints are derived in the form of a hyperparameter space; the aggregated electric vehicle model is:

[0184] (2a)

[0185] where equation (2a) represents the charging power of the aggregated EV subject to a pre-calibration interval through which the aggregated EV obtains the desired charging energy; the hyperparameter matrix H and J are used to describe the sequential charging space:

[0186] (2b)

[0187] where, is a matrix with all permutations of and ; and are column vectors of length with values of 0 and 1, respectively, and further, and are the minimum / maximum charging power and each electric vehicle , and further, and are calculated from the minimum / maximum charging power and of each individual electric vehicle, calculated as follows:

[0188] (2c)

[0189] (2d)

[0190] (2e);

[0191] It is generally required that photovoltaic integration has a minimum power factor, which is adjusted by a smart inverter, which not only provides active power but also absorbs or generates reactive power, the PV photovoltaic model is:

[0192] (3a)

[0193] where: is the photovoltaic power generation at time t ; is the photovoltaic power generation at time tLower limit of photovoltaic power generation at any given time; The upper limit; Power factor; Need to stay Within the range.

[0194] This invention considers three types of distributed power generation: diesel generators, gas turbines, and gas turbines based on carbon capture systems. The carbon emission unit model is as follows:

[0195] (4a)

[0196] (4b)

[0197] (4c)

[0198] (4d)

[0199] in, Let g be the unit commitment variable for generator set g at time t (1 for on, 0 for off). For distributed generation g exist t Output at any moment; and For distributed generation g exist t The lower and upper bounds of the output at each moment; and for t Time generator set g The lower and upper bounds of the active ramp capacity; and Let g be the minimum online time and offline time of generator set g at time t. Formula (4a) represents the active power generation limit; formula (4b) represents the active power ramp-up limit; formulas (4c)-(4d) represent the online time and minimum offline time constraints.

[0200] cement plant direct The main sources are combustion and clinker processes, where the operating conditions are generally constant. This can be considered as linearizing with the production of a cement plant:

[0201] (6a)

[0202] In the formula, This is the direct carbon emission from cement plants. Indicates the carbon emission factor; This indicates the output during the combustion and clinker process, expressed in terms of power consumption. Quadratic function calculation

[0203] (6b)

[0204] wherein, is the consumed power; , and are constant coefficients;

[0205] Direct CE of an electrolytic aluminum plant including fuel combustion CE , raw material consumption CE and electrolytic aluminum production CE :

[0206] (7a)

[0207] determined by the CE properties of the fuel used to produce electrolytic aluminum, calculated by the following equation:

[0208] (7b)

[0209] wherein, f represents the fuel index; represents the average low heat value, net consumption and carbon oxidation rate of the renewable fuel; represents the carbon per unit heat value of the renewable fuel;

[0210] depending on the consumption level of the raw material, i.e. carbon anode, is represented by:

[0211] (7c)

[0212] wherein, represents the consumed power of the electrolytic aluminum plant; , , , and are constant parameters used to describe the raw material consumption when producing electrolytic aluminum;

[0213] from the anode effect in electrolytic aluminum production, which emits two carbon gases CF4 and C2F6, is:

[0214] (7d)

[0215] wherein, and represent the constant CE coefficient, direct cost benefit of a steel plant The four aspects include cost-effectiveness from coal, cost-effectiveness from gas, cost-effectiveness of other steel production processes independent of coal and gas, and cost-effectiveness of steel production, is defined as:

[0216] (8)

[0217] wherein, represents the CE factor of coal, gas, etc. materials; represents the coal, gas required for steel production; represents the steel production and the coal gas production;

[0218] Direct CE of oil refinery Consumption from coke combustion and related raw material processing and transportation:

[0219] (9)

[0220] wherein represents the raw oil consumed by the oil refinery; represents the mass fraction of coke in the oil refining products; represents the utilization efficiency; represents the first i energy and the corresponding CE factor;

[0221] Direct CE of petrochemical plant Caused by three items, including fossil fuel combustion , carbon-containing raw material consumption and CE of calcium carbonate decomposition:

[0222] (10a)

[0223] Determined by the consumed diesel and coke:

[0224] (10b)

[0225] wherein, represents the fossil fuel index; is a constant factor used to describe the first i fuel consumption;

[0226] Depends on the consumption of carbon-containing raw materials:

[0227] (10c)

[0228] wherein represents the index of carbon-containing raw materials; is the carbon content in the carbon raw material; This indicates the consumption of carbon-containing raw materials; This indicates the production of carbides (carbon-containing products) and remediation ash (carbon-containing waste); Indicates the carbon content in carbides and clean ash;

[0229] also, The formula is defined as

[0230] (10d)

[0231] in Indicates the consumption of carbonates; The CE coefficient represents the carbonate and its purity.

[0232] (11)

[0233] In the formula A For commercial building surface area; For the first j Carbon emissions from secondary carbon-containing materials; For the first j CE coefficient of carbon-containing materials; The CE coefficient for the k-th transport method represents the average weight of carbon-containing materials over a distance. Let be the CE coefficient of the h-th energy source and the energy consumption of the demolition process; This refers to the service life and the number of hours the device is used per day.

[0234] Specifically, this also includes constructing a load aggregator model, which is as follows:

[0235] (5a)

[0236] (5b)

[0237] (5c)

[0238] in, For the first d Individual load aggregators t The basic load at any given time; and For the first d Individual load aggregators t Minimum and maximum load at any given time; and For the first d The maximum and minimum demand responses of the load aggregator at time t, and the constraint of the demand response by formula (5a) after the first load aggregator. d The load of a load aggregator; Formula (5b) limits the range of demand response; Formula (5c) represents the load factor requirement.

[0239] In particular, the electricity carbon market trading model is:

[0240] (12a)

[0241] (12b)

[0242] (12c)

[0243] (12d)

[0244] (12e)

[0245] (12f)

[0246] (12g)

[0247] (12h)

[0248] (12i)

[0249] (12j)

[0250] (12k)

[0251] (12l)

[0252] (12m)

[0253] (12n)

[0254] (12o)

[0255] (12p)

[0256] (12q)

[0257] wherein: is the dielectric constant of the node b ; is the conductance of the node b ; is the active power of the node b at time t; is the reactive power of the node b at time t; is the maximum power flow of the distribution line l ; and for a distribution line l at t active / reactive power flow; and for a distribution line l at t square current and node b at t square node voltage magnitude; for a node b at t carbon intensity; for a distribution line l at t carbon intensity; for carbon emissions accompanying power losses along a distribution line t at time l ; for a node b at t indirect carbon emissions; for a node b at t direct carbon emissions; for a node b at t total carbon emissions; for the amount of carbon emission permits purchased from an external carbon emission market; node b at time t traded carbon permits; wherein equations (12a)-(12b) represent node active and reactive power balance; equations (12c)-(12d) constrain bidirectional branch power flow; equation (12e) represents branch voltage drop; equation (12f) represents power flow by second order cone relaxation; equation (12g) represents node voltage range; equation (12h) couples and with power factor; equation (12i) limits the range of ; equation (12j) indicates CE balance, where the right hand indicates total CE from CE units; and the left hand separates the total CE into three parts: CE of power sellers in equation (12j-1), CE of power buyers in equation (12j-2), and CE associated with branch power losses in equation (12j-3); equation (12k) calculates CE associated with branch power losses; equation (12l) calculates node CE intensity by weighted average CE of all injected power; equation (12m) defines carbon intensity of a branch l as carbon emissions from branch lThe carbon intensity of the outflowing node; Formulas (12n)-(12o) calculate the indirect CE of the power sellers and buyers; Formula (12p) adds the indirect CE and the direct CE to obtain the total CE; Formula (12q) indicates that the CE of the bus B should always be lower than its carbon permit.

[0258] The partial observable Markov decision process of discrete time steps manages the electricity-carbon market bid / ask of the DER as a sequential decision process;

[0259] Based on the Markov decision process, each DER performs local training in a federal reinforcement learning model, and each DER derives its own electricity-carbon market transaction decision through a deep Q network.

[0260] According to the types of electricity-carbon market participants, 11 agents are set, including subways, aggregated EVs, PVs, 3 CE units, and 5 load aggregators, namely IZ, CZ, RZ, PZ, and AZ, and the partial observable Markov decision process of discrete time steps is defined by , the state set N , the private observation set S , the action set O , the reward function set A , the state transition function R , and the discount factor T ; gamma ;

[0261] The state set includes the partial observation values of all agents at time t, and according to the DER types, the local observation is described as:

[0262] Subway:

[0263] (13)

[0264] In the formula: is the voltage amplitude of the node b at the moment t ; is the carbon emission intensity of the distribution line l at the moment t ; is the initial carbon permit of the node b at time t; is the operating characteristic of the distribution network; is the transaction information; is the operating parameter of the subway, and here, represents the up / down time of ; represents the ISOE set in the section between the t th station and the th station at time ;

[0265] Other e-carbon actors:

[0266] (14)

[0267] Action set For all agents, the action at time t is defined as:

[0268] Metro:

[0269] (15)

[0270] In the formula: is the electricity trade volume of node b at time t; Node b The transaction carbon permit amount at time t ; , carbon permit trade volume , and operating time and ;

[0271] Other e-carbon actors:

[0272] (16)

[0273] State transition T: through the function , the environment state is associated with the action ; Specifically, the transition of state is determined by the action , as the price solved in the e-carbon market interaction model in (12) is solved by using (12) of the marginal theory. In addition, the state of the metro is converted by the action , for the state ISOEt, it is converted by an internal optimization model, which aims to minimize the ISOE fluctuation from the first to the last intermediate section, which is realized by a portable commercial software;

[0274] Reward function set R: the individualized goal of DER calculated according to the obtained action, the reward function of all DERs has the same composition, but is designed to have specific connotations to adapt to practical applications, which is represented by the following formula:

[0275] (17a)

[0276] In the formula, represents the carbon tax price, which is determined by the carbon intensity and carbon tax The product of the two is calculated; The reward function of DER i is denoted as The transaction utility reflects the individualized target; The carbon emission cost is denoted as The network usage cost is denoted as The bilateral carbon trading cost is denoted as

[0277] Specifically, the transaction utility is modeled as a quadratic function:

[0278] (17b)

[0279] In the formula, for the subway and the aggregated electric vehicles: The energy saving is denoted as , and are all normal numbers, indicating that there is a clear monotonic relationship between and ;

[0280] For the PV photovoltaic: The PV consumption is denoted as and are both negative constants;

[0281] For the carbon emission unit: The power generation cost is denoted as and are both normal numbers;

[0282] For the load aggregator: The negative utility for indicating the expected failure of power consumption is denoted as and are both normal numbers;

[0283] The bilateral carbon trading cost is modeled as a linear function of the carbon permit, for reflecting the transaction credibility, the transaction scale, etc.:

[0284] (17c)

[0285] In the formula: is the bilateral carbon trading cost; is the carbon emission transaction volume between i and m two carbon trading subjects at t time.

[0286] Based on Markov decision process, each DER can perform local training in FRL to achieve individualized objectives during electricity-carbon market trading, where the motivation behind FRL is mainly twofold. First, compared with the communication overhead and scalability issues of existing multi-agent reinforcement learning algorithms, FRL has higher performance, since only periodic updates are required and the training process is distributed among multiple agents, second, unlike centralized training integrated in multi-agent RL that relies on collecting all local information from DERs, FRL allows to protect data privacy by keeping its local data and only sharing model updates. Therefore, FRL is applied to the coupled electricity-carbon market market considering heterogeneous DERs with differentiated objectives, the training process of the federated reinforcement learning model includes global aggregation of updated local models and local model training using actor-critic networks;

[0287] Will I be represented as a set of DERs, Di as a dataset, each DER trains its own local model and passes the model to the electricity-carbon market coordinator to determine the global model , which is represented as Then the global model is broadcast to all DERs for local updates to achieve differentiated objectives of heterogeneous DERs in a self-decision and privacy-protected manner.

[0288] Specifically, WFTD3 is used as a reinforcement learning algorithm for each DER to train its local model in electricity-carbon market trading, which introduces a weighted mean field to improve the overall performance of the traditional version of TD3, in WFTD3, each DER has an actor-critic framework, including an actor network and two critic networks, with parameters of DER i actor network acts in a decentralized manner, taking local observation as input and producing deterministic action , which is then passed to the critic network to output Q-value estimates for action evaluation; the target actor network and the target critic network are incorporated in WFTD3, and their soft update is formulated as:

[0289] (18)

[0290] where is the target update factor, the critic network is updated by temporal difference learning, and the actor network is updated by deterministic policy gradient:

[0291] (19)

[0292] (20)

[0293] To further improve learning performance, the concept of neighbor agents is introduced. In the electric carbon market, agents of the same type, such as different CE cells, are considered to be adjacent to each other. These neighbor agents with similar electric carbon market trading patterns can interact with each other on trading information to make more informed electric carbon market trading decisions. A weighted average field technique is used to describe the effect of neighbor agents on the learning process. i The impact of individual DER behaviors:

[0294] (twenty one)

[0295] In the formula: For network parameters; For sample size; The loss function; for t The target Q value at time t; Discount factor; The state at time t; For actor networks in Actions generated under a given state; The Q-value predicted by the Critic network; The reward that an agent receives from the environment after performing an action; The optimization objective of the Actor network; The gradient with respect to the parameters; for t The state at any given moment; Indicates the first i The weighted action of each DER, Indicates the first i A neighbor agent of a DER, We represent slight perturbations, and then use Taylor's theorem to derive pairwise Q-value functions. :

[0296] (twenty two)

[0297] In the formula, since The first-order terms can be eliminated; the second term serves as the remainder of the Taylor polynomial. ,when When M-smooth, it can be obtained The final approximation.

[0298] Specifically, meta-functions and time-varying control variants are used to reconcile local models trained in WFTD 3. Global model with electric carbon market orchestrator whose main components are meta-functions and time-varying control variants;

[0299] Unlike traditional FL, which seeks to find a model that fits all scenarios, meta-functions aim to find an initialization that has good performance after updates New scenarios, possibly through local gradient updates of only one or a few steps, the function is formulated as:

[0300] (23)

[0301] where, w are the network parameters; β denotes the step size; is the gradient of the meta-function; I is the sample size. The advantage of equation (23) is that it not only maintains the framework of traditional FL, but also captures the differentiated DER, even the difference between DERs within a type, because each DER can use the solution of (23) as an initial point and make slight updates to it to perform well for its own dataset;

[0302] Meta-function is represented by the average of the meta-functions , where the function of the i th DER is defined as:

[0303] (24)

[0304] To update the local model, the gradient of equation (20) is calculated, i.e., the second order of the loss function , which is calculated as follows:

[0305] (25)

[0306] Since the solution of the gradient can be computationally expensive, further development of an unbiased estimate:

[0307] (26)

[0308] where: is the unbiased estimate; denotes the size of ; and is the error in evaluating the model w based on the input data d to predict the true label L ; is calculated by its unbiased estimate through independent batches , and Further solving:

[0309] (27)

[0310] In addition, in order to correct the "client drift" problem in traditional FL, a random control variable is proposed for each DER c i where the average control variable is initialized for the electricity-carbon market coordinator , and the obtained The update of the local model can be defined as:

[0311] (28)

[0312] Then, the random control variable is also updated via the calculated gradient ,

[0313] (29)

[0314] Using formula (24) and formula (25), then aggregate the local updates to update the parameters of the electricity-carbon market coordinator,

[0315] (30)

[0316] Through formula (23)-(30), a round of communication and update between DER and electricity-carbon market coordinator is completed.

[0317] A real 53-node distribution network in a city in China was tested for the proposed electricity-carbon market mechanism, as shown in Figure 5 The DER integrated in the distribution network includes subway, aggregated EV, PV, CE unit (i.e., GT, DG, and CCS-GT unit), load aggregator (IZ, CZ, RZ, PZ, SZ). The specific parameters of these DERs are shown in Table I.

[0318] The minimum / maximum charging rate of the electric vehicle is 0 kW and 8 kW, respectively, and the operation time of the subway is from 6 am to 20 pm. The line length is 3000 m, the OESD capacity is 8.3 kWh, the operation time window is 110-210 s, and the stop time between each station is 40 s.

[0319] In order to prove the advantages of the proposed electricity-carbon market mechanism, the traditional system-centric mechanism and the peer-centric mechanism are compared. The differences between the three markets are as follows:

[0320] 1) Type 1: System-centric mechanism: The electric carbon market is cleared in a centralized manner by the DSO aiming to minimize the total cost of all DERs and the operational cost of the DSO. DERs need to share private electric carbon market trading data with the DSO regardless of differentiated objectives such as joint market.

[0321] 2) Type 2: Peer-centric mechanism: The electric carbon market is cleared in a decentralized manner, allowing DERs to have differentiated objectives to make self-decision. Each DER needs to share private electric carbon market trading data with the DSO to obtain the electric carbon market price, similar to the work in

[23] .

[0322] 3) Type 3: Hybrid mechanism (proposed): An added electric carbon market coordinator (in practice, an electric carbon market industry association or trading hub) coordinates electric carbon market trading by aggregating the implicit electric carbon market parameters, after which each DER obtains a shared model from the electric carbon market coordinator and makes local trading decisions based on its individualized objectives. Each DER needs to share private electric carbon market trading data with the DSO (profit-neutral and independent of the electric carbon market) to obtain the electric carbon market price.

[0323] Figures 6 to 9 The four trading cost items of five representative DERs under the three electric carbon market frameworks are compared. Table I gives the clearing results. From Figures 6 to 9 it can be seen that the system-centric framework has the highest electric carbon market trading cost due to the nearsightedness of the electric carbon market user side. In the peer-centric framework, the overall level of trading cost significantly decreases compared to the system-centric framework. Moreover, in the hybrid electric carbon market framework, since each DER is indirectly affected by the E&C trading decisions of other DERs through the shared model of the electric carbon market coordinator, the trading cost is higher than the peer-centric framework but still lower than the system-centric framework.

[0324] To further verify the effectiveness of the proposed electric carbon market framework, the E&C trading results under the three aforementioned electric carbon market market mechanisms are shown in Figures 10 to 15 From Figures 10 to 12 it can be seen that the electric power trading results of the three power markets have similar compositions. Specifically, the internal generation units, i.e., photovoltaic and CE units, play a dominant role as electric power sellers. The significant difference lies in the dependence on purchased electric power from the external electric power market, where Type 1 exhibits the lowest degree of dependence, Type 2 the highest, and Type 3 occupies an intermediate position. Moreover, from Figures 13 to 15It can be seen that the carbon trading results of the three e-Carbon markets have similar spatial distribution. Carbon emission right buyers are mainly distributed in IZ and some buses equipped with CE devices, while carbon permit sellers are mainly distributed in RZ, PZ and SZ. In particular, the difference between the three e-Carbon markets lies in the purchase of carbon permits from the external carbon market. Similar to electricity trading, carbon trading in Type 1 shows the lightest level of carbon permit purchase, Type 2 is the heaviest, and Type 3 is between the two. Therefore, the results of carbon electricity trading further prove this point.

[0325] To demonstrate the necessity and advantages of combining direct carbon emissions with carbon emissions trading, two cases are designed:

[0326] Case 1: Direct carbon emissions are included in the carbon emissions responsibility of the distribution level of emission reduction carbon trading;

[0327] Case 2: Direct carbon emissions are settled by the external carbon market and excluded from the carbon emissions responsibility of the distribution level of emission reduction carbon trading.

[0328] Figures 16 to 19 The e-Carbon market trading results under these two conditions are shown. Table II lists the economic benefits of the two conditions.

[0329] From Figure 16 and Figure 17 it can be observed that PV consumption in Case 1 is greatly increased compared to Case 2, while CE unit generation is largely suppressed. In addition, it can be seen from Figure 18 and Figure 19 that the important sellers of bilateral carbon trading shift from RA in Case 2 to PV in Case 1, which greatly increases the profitability of PV. In addition, from Figure 18 , Figure 19 and Table II, it can be inferred that the introduction of direct CE in Case 1 significantly increases the carbon trading volume of DER peers at the distribution level by 17.59 tCO 2 compared to Case 2, but reduces the total excess CE from the external carbon market by 40.75 tCO 2. This further leads to a reduction in excess CE cost of 343.44 dollars and an increase in total social welfare of 336.05 dollars. Therefore, considering direct CE is beneficial to decarbonize the electricity market at the distribution level and is crucial to the operation economy.

[0330] In order to highlight the training performance of the proposed pFedScv algorithm, three other personalized FL algorithms are used for comparison, namely,

[0331] 1) FedProx: a proximal term is added in the local subproblem to handle data heterogeneity and is able to merge a variable number of local works.

[0332] 2) FedBN: The batch normalization technique is applied to the client, where the client BN layer and update layer are set to address different degrees of data heterogeneity.

[0333] 3) FedAla: This algorithm adopts an adaptive local aggregation module to determine the initial point of the local Agent by aggregating the global model and the local model into the local target.

[0334] Figures 20 to 24 The evolution of episodic rewards for the four algorithms is shown for five aggregators over 500 sets, where solid lines and shaded areas indicate the moving average over 10 sets and the oscillation of rewards during the training process, respectively. It can be clearly seen that the proposed pFedScv algorithm outperforms the other three algorithms by converging to the highest reward level faster. Taking RZ as an example, the proposed pFedScv algorithm converges to an episodic reward of 110.83, which is 25.14%, 66.55% and 43.86% higher than FedProx, FedBN and FedAla, respectively. Figure 25 The quartile range of the standard deviation of the four algorithms is given, where the median of the standard deviation of the proposed pFedScv algorithm is 74.07%, 87.24% and 52.65% lower than that of the FedProx, FedBN and FedAla algorithms, respectively. This fully verifies the advantages of the stability performance of the proposed pFedScv algorithm. Table III shows the computation time results of the four algorithms over 1000 sets, where the total running time of the proposed pFedScv algorithm is 32.93%, 29.03% and 48.56% lower than that of the FedProx, FedBN and FedAla algorithms, which is due to i) the proximity term in FedProx increases the additional computational complexity; ii) the batch normalization of different data from different DERs makes it difficult for FedBN to converge; iii) aggregating the global model and the local model in FedAla sacrifices the computation time to improve its learning performance. In summary, compared with the FedProx, FedBN and FedAla algorithms, the pFedScv algorithm has overwhelming advantages in terms of fast convergence speed, high reward level, small reward deviation and short computation time.

[0335] The above only describes the preferred embodiments of the present application, and does not limit the present application in any form. Any skilled person in the art can make many possible changes and modifications to the technical solutions of the present application, or modify equivalent embodiments with equivalent changes, without departing from the scope of the technical solutions of the present application. Therefore, any modification, equivalent change and modification of the above embodiments made by the skilled person in the art according to the technical solutions of the present application, without departing from the scope of the technical solutions of the present application, are all within the protection scope of the present application.

Claims

1. A method for electricity-carbon coupling market user-side autonomous decision-making, characterized in that: The method comprises: A carbon electricity market transaction model is constructed, which coordinates carbon electricity sensors through a carbon electricity coordinator, distributes carbon electricity data to different distributed energy resources (DERs), and obtains a shared model from the carbon electricity market coordinator and performs local training, then merges the local training model to generate a global model; the distributed energy resources (DERs) include a subway model, an aggregated electric vehicle model, a PV photovoltaic model, and a carbon emission unit model; A partially observable Markov decision process with discrete time steps is established to manage the electricity-carbon market bidding / asking of the distributed energy resources (DERs) as a sequential decision process; Based on the Markov decision process, each distributed energy resource (DER) performs local training in a federal reinforcement learning model, and each distributed energy resource (DER) derives its own electricity-carbon market transaction decision through local self-training; The subway model is: (1a) (1b) (1c) (1d) (1e) (1f) (1g) (1h) wherein: is the trip index of the metro line; k is the total number of metro stations within a single trip of the metro line; i is the inter-station segment index; T i is the running time of the metro in the i th inter-station segment; ISOE i is the initial energy state of the on-board energy storage device in the i th inter-station segment; Equation (1a) represents the total number of metro stations in operation; Equation (1b) represents the total running time from the first station to the terminal station; Equation (1c) limits the running time of each inter-station segment; Equation (1d) limits the operating ISOE of the OESD for each inter-station trip; Equations (1e)-(1g) are to coordinate the running time and the electricity-carbon market trading period; is the unit electricity-carbon market trading period; and is the running time of the inter-station segment between the first and last stations in the t th electricity trading period; is the running time of the inter-station segment covering the entire section in the t th electricity trading period, calculated as the sum of the running times of the th inter-station segment; t th inter-station segment of the t th trading period is also the last inter-station segment of the t -1th trading period; th inter-station segment t of the +1th trading period; the sum of and of two adjacent trading periods is equal to the running time of the inter-station segment; Equation (1h) represents the net power consumption of the metro during the t th electricity-carbon market trading period, calculated by the function : (1i) wherein and are pre-calibrated constants; The carbon emission unit model is: (4a) (4b) (4c) (4d) (5a) (5b) (5c) (6a) (6b) (7a) (7b) (7c) (7d) (8) (9) (10a) (10b) (10c) (10d) (11) In the formula: Let g be the unit commitment variable for generator set g at time t (1 for on, 0 for off). For distributed generation g exist t Output at any moment; and For distributed generation g exist t The lower and upper bounds of the output at each moment; and for t Time generator set g The lower and upper bounds of the active ramp capacity; and Let g be the minimum online and offline time of generator set g at time t; For the first j Carbon emissions from secondary carbon-containing materials; The carbon content in the carbon feedstock; Formula (4a) represents the active power generation limit; Formula (4b) represents the active power ramp limit; Formulas (4c)-(4d) represent the online time and minimum offline time constraints; For the first d Individual load aggregators t The basic load at any given time; and For the first d Individual load aggregators t Minimum and maximum load at any given time; and For the first d The maximum and minimum demand response of the load aggregator at time t; Formula (5a) limits the demand response after the first... d The load of each load aggregator; Formula (5b) limits the range of demand response; Formula (5c) represents the load factor requirement; This is the direct carbon emission from cement plants. Indicates the carbon emission factor; Indicates the output during the combustion and clinker process; To consume power; , and It is a constant coefficient; Direct carbon emissions from aluminum electrolysis plants; Carbon emissions from fuel combustion Carbon emissions from raw material consumption; Carbon emissions from electrolytic aluminum production; f Indicates fuel index; This indicates the average lower heating value, net consumption, and carbon oxidation rate of renewable fuels; This indicates the carbon content per unit calorific value of renewable fuels; This indicates the power consumption of an electrolytic aluminum plant; , , and It is a constant parameter used to describe the raw material consumption during the production of electrolytic aluminum; and Represents the constant CE coefficient; For the direct cost benefits of steel plants; This indicates the CE coefficient for coal, natural gas, and other materials. This refers to the coal and gas required for steel production; This indicates steel production and gas production; For direct carbon emissions from oil refineries; This indicates the feedstock oil consumed by the refinery; This indicates the mass fraction of coke in refined oil products; Indicates utilization efficiency; Indicates the first i Each energy and the corresponding CE coefficient; Direct carbon emissions from petrochemical plants; Carbon emissions from burning fossil fuels Carbon emissions are generated from the consumption of carbon-containing raw materials. Carbon emissions are generated from the decomposition of calcium carbonate. Indicates the fossil fuel index, It is used to describe the first i The constant coefficient for fuel consumption, An index representing carbon-containing raw materials; This indicates the consumption of carbon-containing raw materials; Indicates the production of carbides and clean ash; Indicates the carbon content in carbides and clean ash; Indicates the consumption of carbonates; The C·E coefficient represents carbonates and their purity; For direct carbon emissions from business centers A For commercial building surface area; For the first j CE coefficient of carbon-containing materials; For the first k The average distance, weight of carbon-containing materials, and carbon emission coefficient of each mode of transportation; for the first... h The energy consumption coefficient (CE) of the secondary energy source and the energy consumption during the dismantling process; the service life and daily operating hours.

2. The electricity-carbon coupling market user-side autonomous decision-making method according to claim 1, characterized in that: The aggregated electric vehicle model is: (2a) wherein equation (2a) represents the charging power of the aggregated EV subject to a pre-calibration interval by which the aggregated EV obtains the desired charging energy; a hyperparameter matrix H and J for describing the sequential charging space: (2b) wherein is a matrix having a number of rows equal to the number of electric vehicles and all permutations of the rows of the matrix ; and are column vectors of length with values equal to 0 and 1, respectively, and, furthermore, and are the minimum / maximum charging power and each electric vehicle , furthermore, and are calculated from the minimum / maximum charging power of each individual electric vehicle, calculated as follows: (2c) (2d) (2e)； In the formula: This represents the upper limit of the aggregated charging power for electric vehicles. This represents the lower bound of the aggregated electric vehicle charging power. For the first n Maximum charging power for electric vehicles; For the first n Lower limit of charging power for electric vehicles; The PV photovoltaic model is: (3a) wherein: is t the photovoltaic power generation at the time; is t the lower limit of the photovoltaic power generation at the time; is the upper limit; is the power factor; needs to be kept in the range .

3. The method of claim 1, wherein: The electricity-carbon market transaction model is: (12a) (12b) (12c) (12d) (12e) (12f) (12g) (12h) (12i) (12j) (12k) (12l) (12m) (12n) (12o) (12p) (12q) In the formula: For nodes b The dielectric constant; For nodes b The electrical conductivity; Let be the active power of node b at time t; Let be the reactive power of node b at time t; For power distribution lines l The biggest trend; and For power distribution lines l exist t The active / inactive current of each moment; and For power distribution lines l exist t Square current at time and node b exist t The squared node voltage magnitude at time t; For nodes b exist t Carbon emission intensity at any given moment; For power distribution lines l exist t Carbon emission intensity at any given moment; Carbon emissions are accompanied by time t Along the power distribution line l Power loss; For nodes b exist t Indirect carbon emissions at any given moment; For nodes b exist t Direct carbon emissions at any moment; For nodes b exist t Total carbon emissions at any given moment; The amount of carbon emission permits purchased from external carbon emission markets; For nodes b In time t The trading carbon permit amount; where, formulas (12a)-(12b) represent the active and reactive power balance of the node; formulas (12c)-(12d) constrain the bidirectional branch power flow; formula (12e) represents the branch voltage drop; formula (12f) represents the power flow through second-order cone relaxation; formula (12g) represents the node voltage range; formula (12h) will and With power factor Coupling; Equation (12i) restriction formula (12j) indicates CE balance, where the right hand indicates total CE from the CE unit; and the left hand splits the total CE into three parts: CE for the power seller in formula (12j-1), CE for the power buyer in formula (12j-2), and CE associated with branch power losses in formula (12j-3); formula (12k) calculates CE associated with branch power losses; formula (12l) calculates node CE intensity by a weighted average CE of all injected power; formula (12m) adds indirect CE and direct CE to obtain total CE for the bus B; formula (12n)-(12o) calculate indirect CE for the power seller and buyer; formula (12p) adds indirect CE and direct CE to obtain total CE; formula (12q) indicates that the CE of the bus B should always be lower than its carbon allowance. l The carbon intensity of a branch is defined as the carbon intensity of the node from which the branch l flows; formula (12n)-(12o) calculate indirect CE for the power seller and buyer; formula (12p) adds indirect CE and direct CE to obtain total CE; formula (12q) indicates that the CE of the bus B should always be lower than its carbon allowance.

4. The method of claim 1, wherein: The portion of the discrete time step observable Markov decision process is through... ,definition, N One agent, state set S Private Observation Set O Action Set A Return function set R State transition function T and discount factor γ ; State set The local observation is described as including all the partial observations of the agents at time t. Subway: (13) wherein: is a node b at t the voltage magnitude at time t; is a distribution line l at t the carbon emission intensity at time t; is the initial carbon allowance for node b at time t; is an operational characteristic of the distribution network; is transaction information; is an operational parameter of the subway, where, denotes the up / down time of denotes the ISOE set for the section between station t and station and station​ Other carbon electricity participants: (14) Action set For all agents, the action at time t is defined as: Subway: (15) In the formula: is the power transaction volume of the node b at time t; is the node b At time t The transaction carbon permit volume; , the carbon permit transaction volume , and the running time And ; Other carbon electricity participants: (16) State transition T: by function implementing, the environmental state is associated with an action ; A set of return functions R: calculate the personalized goals of the distributed energy resources (DERs) according to the obtained actions, and the reward functions of all distributed energy resources (DERs) have the same composition, represented by the following formula: (17a) wherein: represents the carbon tax price, calculated from the carbon intensity and the carbon tax ; and represents the reward function of distributed energy DER i, reflecting the transaction utility of individualized targets; is the carbon emission cost; is the network usage cost; is the bilateral carbon trading cost.

5. The method of claim 4, wherein: The transaction utility Modelled as a quadratic function: (17b) In the formula, for the subway and the polymer electric vehicle: indicates energy saving, , and are normal numbers, indicating and there is a clear monotonic relationship between them; For PV photovoltaic: represents PV consumption, where and are negative constants; For the carbon emission unit: represents the generation cost, where and are normal numbers; For load aggregators: represents a negative utility for representing a power consumption expectation failure, where and are both positive numbers; Bilateral carbon trading costs Modeling as a linear function of carbon permits: (17c) In the formula: Costs associated with bilateral carbon trading; for t time i and m The amount of carbon emissions traded between two carbon trading entities.

6. The method of claim 5, wherein: The training process of the federal reinforcement learning model includes global aggregation of updated local models and actor-critic network used for local model training; Will be I Represented as a collection of distributed energy resources, DERs, represented as datasets, each distributed energy resource, DER, trains its own local model And passes the model to the electric carbon market coordinator to determine the global model Which is represented as The global model is then Broadcast to all distributed energy resources, DERs, for local updates to achieve differentiated objectives for heterogeneous DERs in a self-decision and privacy-protected manner.

7. The method of claim 5, wherein: WFTD3 is used as a reinforcement learning algorithm for each distributed energy DER to train its local model in the electricity-carbon market transaction. In WFTD3, each distributed energy DER has an actor-critic framework, including an actor network and two critic networks, with parameters of the distributed energy DER i of the actor network acting in a decentralized manner, taking local observations as input and producing a deterministic action which is then passed to the critic network to output Q-value estimates for action evaluation; Incorporating target actor networks in WFTD 3 and target critic networks whose soft update formulation is: (18) In the formula is the target update factor, the critic network is updated by the time difference learning, and the actor network is updated by the deterministic policy gradient: (19) (20) The impact of neighbor agents on the behavior of the first i distributed energy DER is described by a weighted mean field technique: (21) where: is the network parameter; is the sample size; is the loss function; is t the target Q value at time t; is the discount factor; is the state at time t; is the action generated by the actor network at state ; is the Q value predicted by the Critic network; is the reward obtained from the environment after the agent performs the action; is the optimization objective of the Actor network; is the gradient of the parameters; is t the state at time t; represents the weighted action of the i th distributed energy resource (DER), represents the neighbor agent of the i th distributed energy resource (DER), represents a slight perturbation, and then the Taylor theorem is used to derive the pairwise Q value function : (22) where, since the first term is eliminated; the second term, as the remainder of the Taylor polynomial when is M-smooth, one obtains the final approximation .

8. The method of claim 5, wherein: Adopting meta-function and time-varying control variants to coordinate local models trained in WFTD 3 and global models of the electricity-carbon market coordinator including meta-function and time-varying control variants; The meta-function is formulated as: (23) wherein: is a network parameter; denotes a step size; is a gradient of a merit function; I is a sample size; merit function is a merit function is represented by an average value, wherein a function of the i-th distributed energy resource (DER) is defined as: i is represented by an average value, wherein a function of the i-th distributed energy resource (DER) is defined as: (24) To update the local model, the gradient of equation (20) is computed which is computed as follows: (25) Further develop unbiased estimates: (26) where: is an unbiased estimate; denotes the size of the evaluation model the error when predicting the true label based on the input data L ; is calculated by an unbiased estimate of the error , and is further solved: (27) a stochastic control variable is proposed for each distributed energy resource DER where the average control variable is initialized for the electrical carbon market coordinator using the obtained the update of the local model is defined as: (28) Then, the stochastic control variable is also updated via the computed gradient , (29) Using formulas (24) and (25), then aggregate local updates to update the parameters of the carbon electricity market coordinator, (30) Through formulas (23)-(30), a round of communication and update between the distributed energy resources (DERs) and the carbon electricity market coordinator is completed.

Images