Power generation alliance electric-carbon-green certificate market bidding decision method based on MADDPG algorithm

Abstract

The application discloses a power generation alliance electricity-carbon-green certificate market bidding decision method based on a MADDPG algorithm. The power generation alliance includes renewable energy units and conventional thermal power units, and the market also includes traditional thermal power generation subjects and wind and light new energy power generation subjects. The method comprises the following steps: based on the selling income of each market subject in the electricity-carbon-green certificate market, a bidding decision model for each market subject participating in the electricity-carbon-green certificate market is constructed as a decision model; the minimum power market electricity purchase cost, the maximum carbon market welfare and the maximum green certificate market welfare are taken as targets respectively, and a power market day-ahead clearing model, a carbon market clearing model and a green certificate market clearing model are constructed as a coupled clearing model; the decision model is taken as an upper model, and the coupled clearing model is taken as a lower model, and a double-layer coupled transaction model is constructed; the MADDPG algorithm is used to solve the double-layer coupled transaction model, and the winning price, the winning amount and the total income of each market subject in the power market, the carbon market and the green certificate market are determined.

Description

Technical Field

[0001] This application relates to the field of energy and power technology, and in particular to a bidding decision-making method for the electricity-carbon-green certificate market of power generation alliances based on the MADDPG algorithm. Background Technology

[0002] The integration of the electricity, carbon, and green certificate markets can reduce carbon emissions while further promoting the development and utilization of renewable energy, providing a fundamental guarantee for efficient resource allocation and stable market operation. To enhance the competitiveness and adaptability of market players, power generation alliances have gradually developed into an important form of improving the efficiency of market player collaboration. By forming power generation alliances, multiple market players can integrate resources, share information, and make collaborative decisions, thereby participating in bidding more efficiently in the electricity, carbon, and green certificate markets. However, in the complex environment of deep multi-market coupling, power generation alliances face many challenges, particularly in how to quickly adjust bidding strategies to cope with the dynamically changing market environment, how to balance resource allocation within the alliance to maximize overall interests, and how to adapt to continuous changes in the market environment. These have become key issues that power generation alliances urgently need to address.

[0003] There is currently a considerable amount of research on the participation of renewable energy power generation entities and thermal power units in the coupled trading of the electricity market, carbon market, and green certificate market. However, the following problems still exist: (1) It does not consider the simultaneous participation of renewable energy power generation entities and fossil energy power generation entities in electricity-carbon-green certificate trading, or it portrays the relationship between renewable energy power generation entities and fossil energy power generation entities in electricity-carbon-green certificate market trading as a perfectly competitive relationship, without considering that the two can obtain cooperative surplus by cooperating in electricity-carbon-green certificate market trading; (2) Most of them assume that in a perfectly information market environment, that is, market participants make decisions based on information about competitors' pricing decisions, which does not conform to the actual market situation.

[0004] Therefore, a new scheme is needed to determine the bidding decisions for power generation alliances to participate in the electricity-carbon-green certificate market. Summary of the Invention

[0005] This application provides a bidding decision-making method for the electricity-carbon-green certificate market of power generation alliances based on the MADDPG algorithm, in an attempt to solve or at least alleviate one of the above-mentioned problems.

[0006] According to one aspect of this application, a bidding decision method for the electricity-carbon-green certificate market of a power generation consortium based on the MADDPG algorithm is provided. The power generation consortium includes renewable energy units and conventional thermal power units, and the market also includes traditional thermal power generation entities and wind and solar renewable energy generation entities. The method includes: constructing a bidding decision model for each market entity participating in the electricity-carbon-green certificate market based on the sales revenue of each market entity in the electricity-carbon-green certificate market, as a decision model. The decision model includes: a bidding decision model for the power generation consortium constructed based on the revenue of the power generation consortium in the electricity market, carbon market, and green certificate market, and the power generation cost; and a bidding decision model constructed based on the revenue of traditional thermal power generation entities in the electricity market and carbon market, and the power generation cost. This paper describes a traditional thermal power generation entity's bidding decision model. Based on the revenue of wind and solar renewable energy generation entities in the electricity market, carbon market, and green certificate market, a bidding decision model for these entities is constructed. With the objectives of minimizing electricity purchase costs in the electricity market, maximizing welfare in the carbon market, and maximizing welfare in the green certificate market, day-ahead clearing models are constructed for the electricity market, carbon market, and green certificate market, respectively, serving as coupled clearing models. A two-layer coupled trading model is constructed, using the decision model as the upper-level model and the coupled clearing model as the lower-level model. The MADDPG algorithm is used to solve the two-layer coupled trading model to determine the winning bid price, winning bid volume, and total revenue for each market entity in the electricity market, carbon market, and green certificate market.

[0007] According to another aspect of this application, a computing device is provided, comprising: one or more processor memories; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, the one or more programs including instructions for performing the methods described above.

[0008] According to another aspect of this application, a computer-readable storage medium is provided for storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform the method described above.

[0009] According to another aspect of this application, a computer program product is provided, including a computer program / instructions, wherein the computer program / instructions, when executed by a processor, implement the method described above.

[0010] In summary, based on the scheme proposed in this application, a two-layer coupled trading model for power generation consortia participating in the electricity-carbon-green certificate market is constructed. The upper-layer model represents the multi-market decision-making model for the power generation consortium and various market participants, while the lower-layer model represents the market clearing model for the electricity-carbon-green certificate market. The MADDPG algorithm is used to solve the two-layer model, obtaining the winning bid price, winning bid volume, and total revenue for each market participant in the electricity market, carbon market, and green certificate market, respectively. This serves as the optimal decision-making scheme for the power generation consortium participating in the electricity-carbon-green certificate market. This scheme can promote the consumption of renewable energy, reduce electricity market clearing bias, and thus achieve a win-win situation for both renewable energy units and fossil fuel units.

[0011] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0012] To achieve the foregoing and related objectives, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings. These aspects indicate various ways in which the principles claimed herein may be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The foregoing and other objectives, features, and advantages of this application will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings. Throughout this application, the same reference numerals generally refer to the same parts or elements.

[0013] Figure 1 This diagram illustrates a framework for power generation consortia to participate in electricity-carbon-green certificate market transactions according to some embodiments of this application;

[0014] Figure 2 A flowchart illustrating a power generation alliance electricity-carbon-green certificate market bidding decision-making method 200 based on the MADDPG algorithm according to some embodiments of this application is shown.

[0015] Figure 3 A schematic diagram of a two-layer coupled transaction model according to some embodiments of this application is shown;

[0016] Figure 4 A schematic diagram illustrating the solution process of a two-layer coupled transaction model according to some embodiments of this application is shown;

[0017] Figure 5 A schematic diagram of load demand according to one embodiment of this application is shown;

[0018] Figure 6 The convergence iteration graphs of different algorithms according to one embodiment of this application are shown;

[0019] Figure 7 A graph showing the revenue changes of various market entities according to one embodiment of this application is provided.

[0020] Figure 8 A schematic diagram of a computing device 800 according to some embodiments of this application is shown. Detailed Implementation

[0021] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0022] The electricity-carbon-green certificate market is a fundamental guarantee for the green, low-carbon, and economically efficient operation of the new power system. As a key player in the system and market, how power generation alliances integrating renewable energy units and conventional thermal power units participate in market bidding is a crucial issue for the operation of the new power system market. Therefore, based on the embodiments of this application, a bidding decision model for power generation alliances based on the multi-agent deep deterministic policy gradient (MADDPG) algorithm is studied under the coupled electricity-carbon-green certificate market.

[0023] First, a framework for power generation alliances to participate in the coupled trading of the electricity-carbon-green certificate market was established.

[0024] In the multi-market coupled trading process of electricity, carbon, and green certificates, power generation alliances, as well as traditional thermal power generation entities (i.e., traditional thermal power units) and wind and solar renewable energy generation entities (i.e., wind and solar renewable energy units) participating in market transactions, need to comprehensively consider the price information and trading volume of each market. By formulating scientific and reasonable trading strategies, they can maximize both energy and environmental value benefits in the multi-market electricity-carbon-green certificate trading. This multi-market coupled trading model places higher demands on the decision-making capabilities of market participants. In particular, how to coordinate the relationship between economic interests in the electricity market and environmental benefits in the carbon and green certificate markets becomes crucial for power generation alliances to optimize their trading strategies.

[0025] To reduce the complexity of the trading model, this application unifies the trading timescales of the electricity market, green certificate market, and carbon market, adopting the trading timescale of the spot market for modeling and analysis. First, the power generation alliance needs to consider its own clearing decision in the electricity market. By accurately predicting electricity market prices and demand, it can formulate a reasonable clearing plan to maximize its profits in the electricity market. Second, after clearing in the electricity market, the power generation alliance needs to realize its environmental value in the carbon market and green certificate market. At this stage, special attention must be paid to avoiding double-counting of environmental value. Simultaneously, by combining the price changes and trading mechanisms of the carbon market and green certificate market, an optimized trading strategy should be formulated to ensure the best returns in realizing environmental value. Finally, the power generation alliance needs to engage in game theory with traditional thermal power generation entities and wind and solar renewable energy generation entities in the multi-market trading of electricity, carbon, and green certificates. It must fully consider the strategic behavior of other market participants and maximize its own profits through competition and cooperation.

[0026] Based on the above analysis, this application proposes a framework for power generation consortia to participate in multi-market trading of electricity, carbon, and green certificates, and systematically studies its decision-making process. This framework is based on electricity market clearing, combines the realization of environmental value in the carbon and green certificate markets, and comprehensively considers the game-theoretic behavior of other market participants, striving to provide efficient and scientific trading strategy support for power generation consortia under multi-market coupling conditions. The specific framework is as follows: Figure 1 As shown.

[0027] Combination Figure 1 Thermal power generators, wind power generators, solar power generators, and power generation alliances supply electricity, while users generate demand for electricity, which is then traded in the electricity market. The flow of electricity in the electricity market is as follows: Figure 1 As shown by the blue dashed line. The carbon market can rationally allocate emission reduction resources within a specific scope and reduce the cost of greenhouse gas emission reduction. It achieves a trading mechanism for reducing carbon emissions by establishing legal carbon emission rights and allowing these rights to be bought and sold. The flow of carbon allowances in trading is as follows: Figure 1 The gray dashed lines indicate the flow of CCERs (Certified Emission Reductions) in the carbon market. Figure 1 As shown by the orange dashed line. The green certificate market allows for market transactions using green certificates as the underlying asset through a green certificate trading platform. The flow of green certificates in the green certificate market is as follows: Figure 1 The green dotted line indicates the market participants, including wind power generators, photovoltaic power generators, (electricity) users, and electricity sales companies (not shown). The main purpose of the green certificate market is to promote the market-based consumption of new energy power generation and reduce the burden of national financial subsidies.

[0028] The synergy of the electricity-carbon-green certificate market needs to be achieved in the following ways:

[0029] (1) Collaboration of Market Decision-Making Entities. Thermal power generators and renewable energy generators (i.e., wind power generators and photovoltaic power generators) supply electricity, while users generate demand for electricity, thus creating green electricity prices and thermal power prices in the electricity market. Carbon prices, CCER prices, and green certificate prices share commonalities in signaling environmental value. Linking carbon trading mechanisms and green certificate trading mechanisms to ensure the uniqueness of green environmental rights helps to collaboratively promote green transformation.

[0030] (2) Coordinated Multi-Market Mechanisms. China's carbon market operates under constraints such as total carbon emissions and carbon offset ratios. Controlled-emission power generation enterprises further determine their carbon quota trading strategies based on government-allocated quotas. The CCER offset mechanism expands market participants and reduces compliance costs for controlled-emission entities. Carbon quota prices and CCER prices are formed through secondary market trading. The green certificate trading market operates under the constraints of renewable energy power consumption responsibility and penalties. Renewable energy power generators certify and hold green certificates, and renewable energy consumption responsibility entities purchase green certificates, thereby forming green certificate prices.

[0031] (3) Coupling and Coordination of Trading Times in Multiple Markets. Currently, the electricity market and the green certificate market operate independently. The trading scale of the electricity market is mainly annual, monthly, daily, and hourly, while the rules of the green certificate market do not specify the trading scale. However, the number of green certificates generated is linked to the actual trading volume in the electricity spot market, and the trading time scale of green certificates can be matched with the trading time scale of the spot market. The coupling of trading times between the electricity market and the carbon market includes the coupling of market trading links and the coupling of quota settlement links. The initial carbon quota allocation and quota settlement of conventional thermal power units are carried out on an annual basis, and carbon emission rights are traded in the carbon market on a daily time scale. The trading time scale of the carbon market can also be matched with the trading time scale of the spot market.

[0032] According to some embodiments of this application, the power generation consortium consists of conventional thermal power units and renewable energy units (such as wind and solar power units), and its decision-making process is significantly influenced by a multi-market coupling environment. In the electricity-carbon market, the carbon allowance usage of thermal power units directly determines their carbon trading costs: insufficient allowances require purchase, increasing operating costs; surplus allowances can be sold, generating additional revenue. Meanwhile, renewable energy units participate in carbon market trading through Certified Emission Reductions (CCERs), and their revenue is influenced by both carbon market demand and policy restrictions. In the electricity-green certificate market, the power generation of renewable energy units determines the supply of green certificates, while the green electricity consumption obligations of electricity retailers constitute the main demand. The power generation consortium needs to weigh the benefits between the carbon market and the green certificate market, while considering policy constraints on initial allowance allocation rules, CCER offset ratio caps, and green certificate application and trading rules.

[0033] In the context of a multi-market coupling environment encompassing electricity, carbon, and green certificates, the decision-making complexity of power generation alliances is further amplified. There is a linkage between the carbon market and the green certificate market; carbon price fluctuations affect the operating costs of thermal power units, while green certificate prices are directly influenced by fluctuations in renewable energy generation and market demand. Power generation alliances also need to dynamically adjust internal resource allocation and integrate the output characteristics of thermal power units and renewable energy units to meet multiple objectives such as carbon emission reduction, green electricity consumption, and stable power supply. Power generation alliances need to comprehensively analyze the supply and demand relationships and price fluctuations of the three major markets and flexibly adjust bidding strategies to cope with changes in market rules and policy constraints. Therefore, in the bidding decisions of power generation alliances participating in the electricity-carbon-green certificate market, the complexity and potential benefits brought about by multi-market coupling need to be fully considered. By establishing a collaborative optimization model, power generation alliances can find a balance between carbon market revenue, green certificate market revenue, and electricity market clearing price revenue, thereby maximizing overall benefits. At the same time, the power generation alliance also needs to maintain its competitive advantage in the competition with other market players. Through scientific decision-making optimization, it can not only achieve continuous improvement in its own revenue, but also play a positive role in promoting the development of renewable energy, optimizing resource allocation, and promoting the achievement of the national dual-carbon goals.

[0034] Based on this, this application proposes a bidding decision-making method for the electricity-carbon-green certificate market based on the MADDPG algorithm. This method is suitable for implementation in the electricity-carbon-green certificate market, where market participants include: traditional thermal power generation entities, wind and solar renewable energy generation entities, and power generation alliances. The power generation alliances further include renewable energy units and conventional thermal power units. Figure 2 The diagram illustrates a flowchart of a power generation alliance electricity-carbon-green certificate market bidding decision-making method 200 based on the MADDPG algorithm according to some embodiments of this application.

[0035] like Figure 2 As shown, Method 200 begins with S210. Based on the sales revenue of each market participant in the electricity-carbon-green certificate market, a pricing decision model for each market participant in the electricity-carbon-green certificate market is constructed as the decision model.

[0036] In some embodiments, the decision-making model includes: a bidding decision model for a power generation alliance based on the revenue and generation cost of the power generation alliance in the electricity market, carbon market, and green certificate market, respectively; a bidding decision model for a traditional thermal power generation entity based on the revenue and generation cost of the traditional thermal power generation entity in the electricity market and carbon market, respectively; and a bidding decision model for a wind and solar new energy power generation entity based on the revenue of the wind and solar new energy power generation entity in the electricity market, carbon market, and green certificate market, respectively.

[0037] The following sections will introduce the pricing decision-making models for the three market participants.

[0038] • Bidding decision model of power generation alliance

[0039] The power generation consortium's revenue comes from the electricity market, carbon market, and green certificate market. Electricity market revenue comes from the sale of electricity, carbon market revenue from the sale of carbon allowances, and green certificate market revenue from the sale of green certificates. In the electricity spot market, the variable costs of wind and solar renewable energy generation are very small and can be ignored. This pricing model reflects its decision-making behavior in participating in the electricity-carbon-green certificate market, aiming to maximize its own revenue.

[0040] Specifically, the process of building a pricing decision model for a power generation consortium involves the following five steps.

[0041] The first step is to take into account the uncertainty of wind and solar power output, generate multiple preset wind and solar power output scenarios, and calculate the probability of each scenario occurring.

[0042] Because the output of solar and wind power is subject to random fluctuations, the actual output of renewable energy power producers often deviates from the cleared electricity volume in the spot market, which is then used for performance evaluation. When renewable energy power producers form a power generation alliance with fossil fuel power producers (e.g., conventional thermal power units), they are given priority in dispatch based on the cleared electricity volume in the spot market. The portion of their output exceeding the cleared electricity volume can substitute for a portion of fossil fuel power. If there is a negative deviation, the fossil fuel units will make up the difference, effectively reducing the performance evaluation costs for renewable energy power producers. After the power generation alliance is formed, fossil fuel power producers need to provide a safety net for the uncertainty of renewable energy output. While they may reduce output when renewable energy units generate more electricity, the green certificates provided by renewable energy power producers can reduce their costs of purchasing green certificates to meet their consumption obligations, thus allowing them to benefit from the cooperation. The CCERs (China Carbon Emission Reduction) for renewable energy can offset part of the power generation alliance's carbon allowances, reducing its carbon purchase costs in the carbon market.

[0043] In some embodiments, due to the uncertainty of wind and solar power output, a probability distribution model is first used to simulate the distribution of wind speed and solar intensity, generating power output curves for wind speed and solar intensity, respectively. See below.

[0044] Assuming the wind speed v follows a Weibull distribution, its probability density function is expressed as:

[0045]

[0046] In the formula: k>0 is the shape parameter, and λ>0 is the scaling parameter.

[0047] Assuming the light intensity I follows a Beta distribution, its probability density function is expressed as:

[0048]

[0049] In the formula: I maxThe maximum value of I at time t, in kWh / m³. 2 α and β are shape parameters.

[0050] Subsequently, based on the wind speed output curve and the light intensity output curve, a first number of preset wind and solar power output scenarios are generated, and the probability of each wind and solar power output scenario is calculated according to the relationship between wind speed, light intensity and actual wind and solar power output.

[0051] In some embodiments, Latin hypercube sampling is used to generate w sets of wind and solar power output scenarios according to a normal distribution (the value of w is, for example, 1000). The scenario reduction method is used to obtain n typical wind and solar power output scenarios (the value of n is, for example, 10). Finally, the probability of each wind and solar power output scenario occurring is calculated based on the relationship between wind speed, light intensity and actual wind and solar power output.

[0052] The second step involves combining various wind and solar power output scenarios and, based on the power generation alliance's revenue in the electricity market, carbon market, and green certificate market, as well as its power generation costs, using the maximization of the power generation alliance's total revenue as the objective function.

[0053] In some embodiments, the total revenue of the power generation consortium is obtained as follows: For each wind and solar power output scenario, the electricity market revenue is determined based on the day-ahead energy market clearing price and the power generation consortium's winning bid volume in the day-ahead energy market; for each wind and solar power output scenario, the carbon market revenue is calculated based on the carbon market clearing price and the power generation consortium's winning bid volume in the carbon market; for each wind and solar power output scenario, the green certificate market revenue is calculated based on the green certificate market clearing price and the power generation consortium's winning bid volume in the green certificate market; then, the total revenue of the power generation consortium is obtained by subtracting the power generation cost of conventional thermal power units in the consortium from the sum of the electricity market revenue, carbon market revenue, and green certificate market revenue.

[0054] The specific expression for the objective function of the power generation alliance is as follows:

[0055]

[0056] in,

[0057]

[0058] In the formula, F GA This represents the total revenue of the power generation consortium; This indicates the power generation alliance's earnings in the day-ahead market; This indicates the benefits that the power generation alliance receives in the carbon market; This indicates the power generation consortium's revenue in the green certificate market; π represents the power generation cost of the h-th conventional thermal power unit in the power generation consortium; H represents the number of conventional thermal power units in the power generation consortium; sThe probability of scenario s, which contributes to the wind and solar power, occurring; S represents the number of such scenarios. This represents the energy market clearing price at time t. λ represents the amount of electricity won by the power generation consortium in the day-ahead energy market at time t; ca To clear prices in the carbon market; The amount of electricity generation alliances won in the carbon market; λ TGC Clearing out prices in the green certificate market; The amount of green certificates won by the power generation consortium in the market; a h b h c h The cost coefficient for thermal power units is T = 24, t = 1, 2, ..., 24.

[0059] The third step is to generate the reporting constraints for the power generation alliance based on the constraints of electricity volume reporting in the electricity market, carbon quota reporting in the carbon market, and green certificate reporting in the green certificate market.

[0060] The specific formula is shown below, where the three expressions represent the constraints on electricity declaration, carbon quota declaration, and green certificate declaration, respectively:

[0061]

[0062] In the formula, This indicates the amount of electricity generated by the power generation alliance reported in the day-ahead energy market at time t; and This represents the minimum and maximum amount of electricity won by the power generation consortium in the day-ahead energy market at time t; For the carbon emissions of the power generation alliance; This represents the amount of electricity generation alliances have won bids in the carbon market; The total carbon allowance for the power generation consortium; This represents the amount of green certificates won by the power generation consortium in the green certificate market. This indicates the number of green certificates obtained by the power generation consortium.

[0063] Specifically, the carbon allowance declaration constraints include: the amount of carbon allowances won by the power generation consortium in the carbon market must not be less than the carbon emissions of conventional thermal power units within the consortium, and must not exceed the total carbon allowances obtained by the consortium. The total carbon allowances obtained by the consortium include: carbon allowances obtained by conventional thermal power units within the consortium, and carbon allowances obtained by renewable energy units within the consortium through certified voluntary emission reductions.

[0064] In this embodiment, a baseline method is used for free allocation. For the power generation consortium mentioned in this paper, the carbon emission sources are mainly conventional thermal power units. Meanwhile, the renewable energy units within the consortium can participate in CCERs (China Certified Energy Emission Reductions). Assuming that one CCER is equal to one carbon allowance, the total carbon allowance obtained by the power generation consortium is expressed as:

[0065]

[0066] in,

[0067]

[0068] π represents the total carbon allowance obtained by the power generation consortium at time t. s The probability of scenario s, which contributes to the wind and solar power, occurring; S represents the number of such scenarios. Carbon allowances obtained by conventional thermal power units in the power generation consortium; κ1 represents the carbon allowance obtained by renewable energy generating units in the power generation alliance through certified voluntary emission reductions; κ2 represents the carbon emission coefficient per unit of electricity; and κ3 represents the CCER conversion coefficient per unit of electricity. Let t be the power generation of conventional thermal power units in the power generation alliance at time t; Let t be the power generation of renewable energy generating units in the power generation alliance at time t; a is the ratio of renewable energy units in the power generation alliance that are selected to convert into certified voluntary emission reductions; H and E are the number of conventional thermal power units and renewable energy units in the power generation alliance, respectively; T = 24, t = 1, 2, ..., 24.

[0069] Since the carbon emissions of a generating unit are directly proportional to its output, the carbon emissions of conventional thermal power units in the power generation consortium are expressed as follows:

[0070]

[0071] in, For the carbon emissions of the power generation consortium; π s The probability of scenario s occurring as a landscape power-generating scene; S represents the number of landscape power-generating scenes; κ GH Carbon emission coefficient; Let H be the output of the h-th conventional thermal power unit in the power generation alliance at time t; H is the number of conventional thermal power units in the power generation alliance; T = 24, t = 1, 2, ..., 24.

[0072] The constraints on the number of green certificate applications include: the number of green certificate bids won by the power generation consortium in the green certificate market shall not exceed the number of green certificates obtained by the power generation consortium. The number of green certificates obtained by the power generation consortium is expressed as follows:

[0073]

[0074] in, Indicates the number of green certificates obtained by the power generation consortium; π s The probability of scenario s occurring as a landscape power-generating scene; S represents the number of landscape power-generating scenes; κ TGC For new energy green certificates and electricity conversion coefficient; The day-ahead market winning bid volume of new energy generating units in the power generation alliance; E represents the number of new energy generating units in the power generation alliance; T = 24, t = 1, 2, ..., 24.

[0075] The fourth step is to generate the bidding constraints for the power generation alliance in the electricity market, carbon market, and green certificate market.

[0076] The power generation alliance's bids in the electricity market, carbon market, and green certificate market should also be within a certain range. The bidding constraints for its participation in each market are shown in the following formula:

[0077]

[0078] In the formula, These are the power generation alliance's bids in the electricity market, carbon market, and green certificate market, respectively. These represent the upper and lower limits of the power generation alliance's bids in the electricity market, carbon market, and green certificate market, respectively.

[0079] The fifth step is to construct a pricing decision model for the power generation alliance by utilizing the objective function, quantity reporting constraints, and pricing constraints of the power generation alliance.

[0080] Bidding decision-making model for traditional thermal power generation entities

[0081] Similarly, the bidding decision model for traditional thermal power generation entities includes the objective function of the traditional thermal power generation entity and the corresponding quotation constraints and pricing constraints. For details, please refer to the bidding decision section of the conventional thermal power units in the aforementioned power generation alliance. Specifically, the total revenue of the traditional thermal power generation entity is the sum of the revenue of the traditional thermal power generation entity in the electricity market (refer to the aforementioned formula (4)) and the revenue in the carbon market (refer to the aforementioned formula (5)), minus the power generation cost of the traditional thermal power generation entity (refer to the aforementioned formula (7)). In other words, compared with the aforementioned formula (3), the objective function of the traditional thermal power generation entity does not include the revenue in the green certificate market.

[0082] Similarly, the reporting constraints for traditional thermal power generation entities only need to include: electricity reporting constraints and carbon quota reporting constraints; the bidding constraints for traditional thermal power generation entities only need to include: bidding constraints in the electricity market and carbon market. For specific expressions, please refer to the relevant content in the aforementioned bidding decision model of the power generation alliance.

[0083] Bidding decision model for wind and solar new energy power generation entities

[0084] Similarly, the bidding decision model for wind and solar renewable energy power generation entities includes the objective function of the wind and solar renewable energy power generation entities and the corresponding quotation constraints and pricing constraints. For details, please refer to the renewable energy unit bidding decision section in the aforementioned power generation alliance. Specifically, the total revenue of wind and solar renewable energy power generation entities is the sum of the revenue of wind and solar renewable energy in the electricity market (refer to the aforementioned formula (4)), the revenue in the carbon market (refer to the aforementioned formula (5)), and the revenue in the green certificate market (refer to the aforementioned formula (6)). That is to say, compared with the aforementioned formula (3), the objective function of wind and solar renewable energy power generation entities does not include the power generation cost.

[0085] When determining the reporting constraints for wind and solar new energy power generation entities, the constraints on electricity reporting and the number of green certificates reported can refer to the relevant constraints of the power generation alliance, but the constraints on carbon quota reporting are as follows: In other words, the total carbon allowance of wind and solar new energy power generation entities is the CCER obtained by wind and solar new energy power generation entities.

[0086] The pricing constraints for wind and solar new energy power generation entities can refer to the pricing constraints of the power generation alliance.

[0087] Subsequently, in S220, with the objectives of minimizing electricity purchase costs in the electricity market, maximizing welfare in the carbon market, and maximizing welfare in the green certificate market, a day-ahead clearing model for the electricity market, a clearing model for the carbon market, and a clearing model for the green certificate market are constructed as coupled clearing models.

[0088] The following sections introduce the clearing models for the three markets.

[0089] Electricity market clearing model

[0090] With the goal of minimizing electricity purchase costs, the SCUC (Security Constrained Unit Commitment) and SCED (Security Constrained Economic Dispatch) models are used to clear the market and obtain the unit start-up and shutdown plans, the day-ahead output of the units, and the day-ahead node electricity price.

[0091] The SCUC model is the core model for power system dispatching, primarily used to determine the output and start-up / shutdown times of each generating unit in the electricity market, as well as the power allocation within the transmission network, to achieve the safe and stable operation of the power system. The SCUC model determines the output and start-up / shutdown times of generating units by minimizing the electricity purchase cost, while also considering the security constraints of the electricity market. In this embodiment, the objective function of SCUC is as follows:

[0092]

[0093] In the formula, F1 represents the total cost of the SCUC model (i.e., the cost of purchasing electricity); This indicates the effort cost of each entity in the day-ahead market; O STR This represents the start-up and shutdown costs of traditional thermal power generation systems. These are the day-ahead market quotes at time t for power generation alliances, traditional thermal power generators, and wind and solar renewable energy generators. These represent the winning bid amounts at time t for the power generation alliance, traditional thermal power generation entities, and wind and solar new energy power generation entities, respectively; U m,t Let M be a 0-1 variable representing the start-up / shutdown state of the traditional thermal power generation unit m at time t, where 1 represents start-up and 0 represents shutdown; M is the number of traditional thermal power generation units; N is the number of wind and solar renewable energy generation units; S m Let m be the start-up and shutdown cost of the traditional thermal power generation unit; T = 24, t = 1, 2, ..., 24.

[0094] In addition, the constraints of the SCUC model mainly include: power balance constraints, generator output constraints, generator ramping constraints, generator start-up and shutdown constraints, and power flow constraints. These are explained in detail below (it should be noted that the parameters in the following formulas can be found in the relevant descriptions above; repeated descriptions will not be repeated).

[0095] Power balance constraints:

[0096]

[0097] In the formula, The total load demand of the market at time t under the wind and solar power output scenario s.

[0098] Generator output constraints:

[0099]

[0100] Generator set ramping constraints:

[0101]

[0102] In the formula, These represent the uphill and downhill ramp rates for conventional thermal power units, respectively.

[0103] Unit start-up and shutdown constraints:

[0104]

[0105] In the formula, These are the minimum start-up and shutdown times for a conventional thermal power unit m, respectively. These represent the time periods during which unit m has been continuously started and shut down in time period t.

[0106] Current constraints:

[0107]

[0108] In the formula, F l,max For the maximum permitted capacity of the line; ρ i,l , ρ n,l These are the power generation and load transfer distribution factors for this node, respectively.

[0109] The SCED model is an economic dispatch model for power systems, primarily used to determine the output of each generating unit in the electricity market and to minimize total cost while meeting electricity demand. The SCED model determines the output of generating units by minimizing total cost, taking into account the security constraints of the electricity market. In this embodiment, the objective function of SCED is as follows (it should be noted that the parameters in the following series of formulas can be referred to in the relevant descriptions above; repeated parts will not be repeated):

[0110]

[0111] In addition, the constraints of the SCED model mainly include: power balance constraints, generator output constraints, generator ramping constraints, power flow constraints, and other constraints. Specific explanations are as follows.

[0112] Power balance constraints:

[0113]

[0114] Generator output constraints:

[0115]

[0116] Generator set ramping constraints:

[0117]

[0118] In the formula, These represent the uphill and downhill ramp rates for conventional thermal power units, respectively.

[0119] Current constraints:

[0120]

[0121] In the formula, F l,max For the maximum permitted capacity of the line; ρ i,l , ρ n,l These are the power generation and load transfer distribution factors for this node, respectively.

[0122] Meanwhile, based on the day-ahead market SCED calculation results, the Lagrange multipliers of the power market load balance constraints and branch power flow constraints for each time period are obtained. Therefore, the nodal price of node k in time period t can be expressed as:

[0123]

[0124] In the formula, Let λ be the nodal electricity price of node k in time period t. t For power balance constraints, Lagrange multipliers Let G represent the Lagrange multipliers for the maximum forward power flow constraint and the maximum reverse power flow constraint of the branch, respectively. l-k This represents the generator output power transfer distribution factor from node k to branch l.

[0125] In summary, by calling the SCUC model and the SCED model, we can obtain the unit start-up and shutdown plan, the day-ahead output of the units, and the day-ahead node price.

[0126] Carbon market clearing model

[0127] Assume that the carbon market is centrally cleared by a carbon market exchange. The clearing process of the carbon market aims to maximize market welfare, as shown in the following expression (it should be noted that the parameters in the following series of formulas are explained in the context; repeated details will not be elaborated upon):

[0128]

[0129] In the formula, These represent the amount of carbon allowances purchased by power generation entity l in the carbon market and the market bid, respectively. These represent the amount of carbon allowances purchased and the market bidding by other industry market participants in the carbon market, respectively. These represent the amount of carbon allowances sold by power generation entity l in the carbon market and the market bid, respectively. These represent the amount of carbon allowances sold by the power generation consortium in the carbon market and the market bidding, respectively.

[0130] In addition, the carbon market clearing model includes several constraints. First, there is a market carbon allowance balance constraint, as shown in the following expression:

[0131]

[0132] Where: μ c To clear prices in the market.

[0133] In addition, the constraints also include: market carbon quota reporting constraints, the quantity of carbon quotas purchased by power generators and other market participants is non-negative, and the quantity of carbon quotas sold by power generators and power generation alliances is no greater than the quantity they hold.

[0134]

[0135] In the formula, C lt Clm , These are the maximum reporting limits for power generation entities, power generation alliances, and other industry entities in the carbon market.

[0136] • Green Certificate Market Clearing Model

[0137] Assume the green certificate market is centrally cleared by a green certificate market exchange. The clearing of the green certificate market has the objective function of maximizing market welfare, and its expression is as follows (it should be noted that the parameters in the following series of formulas can be referred to the relevant descriptions in the context; repeated parts will not be elaborated again):

[0138]

[0139] In the formula: These represent the amount of carbon allowances purchased by power generation entity l in the carbon market and the market bid, respectively. These represent the amount of carbon allowances purchased and the market bidding by other industry market participants in the carbon market, respectively. These represent the amount of carbon allowances sold by power generation entity l in the carbon market and the market bid, respectively. These represent the amount of carbon allowances sold by the power generation consortium in the carbon market and the market bidding, respectively.

[0140] In addition, the green certificate market clearing model includes several constraints. First, there is a constraint on the balance of the number of green certificates in the market:

[0141]

[0142] In addition, the constraints also include: market green certificate reporting constraints, and that the quantity of green certificates purchased by power generators and other market participants must be non-negative, while the quantity of green certificates sold by power generators and power generation alliances must not exceed the quantity they hold.

[0143]

[0144] In the formula: C lt C lm , These are the maximum reporting limits for power generation entities, power generation alliances, and other industry entities in the green certificate market.

[0145] Subsequently, in S230, a two-layer coupled trading model is constructed, with the decision model as the upper-layer model and the coupled clearing model as the lower-layer model.

[0146] In summary, the upper-level models include: the objective functions and constraints of the power generation alliance, the objective functions and constraints of traditional thermal power generation entities, and the objective functions and constraints of wind and solar renewable energy generation entities. The lower-level models include: the objective functions and constraints of electricity market clearing (SCUC model and SCED model), carbon market clearing, and green certificate market clearing.

[0147] Figure 3 A schematic diagram of a two-layer coupled transaction model according to some embodiments of this application is shown. Figure 3 In this system, conventional thermal power units represent the main body of traditional thermal power generation, while renewable energy units represent the main body of wind and solar renewable energy generation. First, power generation alliances, renewable energy generating units, and conventional thermal power units submit electricity price and quantity declarations in the day-ahead market. The trading center conducts centralized clearing based on the declared information, and the dispatch center performs safety verification to determine the start-up and shutdown status and power output of conventional thermal power units and those within the power generation alliance. When power generation alliances and renewable energy units submit bids in the day-ahead market, the deviation between the predicted and actual output of wind and solar power needs to be considered. Based on the clearing results of the day-ahead market, the number of green certificates, CCERs, and carbon quota requirements for conventional thermal power units are determined for the power generation alliances and wind / solar renewable energy units. They then participate in the clearing of the carbon market and green certificate market, calculating their returns from participating in these markets. Based on the trading process of the electricity-carbon-green certificate market, a two-layer coupled trading model is designed. The upper-layer model, combined with deep reinforcement learning algorithms, reflects the individual rationality of power generation alliances participating in the market, while the lower-layer model reflects the trading mechanism, coupling relationship, and trading process of the electricity-carbon-green certificate market.

[0148] Subsequently, in S240, the MADDPG algorithm is used to solve the two-layer coupled trading model to determine the winning bid price, winning bid volume, and revenue of the power generation consortium in the electricity market, carbon market, and green certificate market, respectively.

[0149] The MADDPG method is used to solve the model. On the one hand, it can solve the model in a multi-dimensional continuous price and market state space. On the other hand, the multi-agent reinforcement learning method can reflect the competitive relationship between multiple market participants and is one of the effective algorithms for solving the problem of incomplete information market game.

[0150] Reinforcement learning is widely used in decision optimization problems. It is an interactive learning method based on Markov decision processes. Its iterative process is as follows: the agent interacts with the environment to maximize the long-term reward r. Based on the current state s, it takes action a. Then, the environment returns a reward to the agent based on the chosen action and updates the state. This process is repeated until the optimal action selection policy π is obtained. *(a|s). MADDPG is an extension of the DDPG algorithm. Its main innovation lies in the algorithmic framework of centralized training and distributed execution among multiple agents, which solves the problem of environmental non-stationarity in multi-agent systems.

[0151] MADDPG adopts the actor-critic network from the DDPG algorithm, but with the difference that each agent contains a set of actor-critic networks. The actor network collects local information and makes action selections, while the critic network collects global information and guides parameter updates. Specifically, when there are N agents in a system, its policy set π = {π1, ..., π} N} by θ={θ1,…,θ N}Parameterization, in this case, the expected reward gradient can be expressed as:

[0152]

[0153] In the formula, This is a centralized action-value function that takes the actions and state information of all agents as input and outputs the Q-value for each agent. The new state information includes the observations of all agents, o = {o1, ..., o2}. N}, D={s,s′,a1,…,a N ,r1,…,r N} is the experience replay pool, used to record the results of previous training.

[0154] The method for updating parameters is as follows:

[0155]

[0156] In the formula, μ is the set of policy networks, representing the mapping from the state space to the action space; It has a delay coefficient θ′ i The set of update strategies.

[0157] The above formula assumes that each agent knows the strategies of other agents, but in the case of incomplete information, it is necessary to... Approximating the strategies of other agents, the more refined equations are as follows:

[0158]

[0159] In the formula, is an approximate parameter; H represents the entropy of the policy distribution.

[0160] Figure 4 A schematic diagram illustrating the solution process of a two-layer coupled transaction model according to some embodiments of this application is shown. Combined with... Figure 4The MADDPG algorithm is used to solve the two-layer coupled trading model. This involves using market participants as agents for the MADDPG algorithm, with the electricity-carbon-green certificate market as the algorithm's environment, and iteratively combining the decision model and the coupled clearing model until the optimal action selection strategy is obtained. The optimal action selection strategy includes the winning bid results and the corresponding bid decisions submitted by each market participant.

[0161] Specifically, the bidding and quantity decision-making process of market participants is the action of an agent, the market clearing result is the state of the environment, and the net profit of each market participant is the agent's reward. Combined with... Figure 4 The specific correspondence is as follows.

[0162] (1) State space

[0163] In the MADDPG algorithm, the state space s consists of the observations o of each agent. i The composition, in this model, corresponds to the winning bid results and known market information for each market participant; therefore, the state set is represented as:

[0164] s = {o1, o2, ... o N} (37)

[0165]

[0166] In the formula, These represent the winning bid prices of agent i in the electricity market, carbon market, and green certificate market, respectively. These represent the winning bids of agent i in the electricity market, carbon market, and green certificate market, respectively.

[0167] (2) Action space

[0168] The action space α consists of the actions of each agent, that is, the set of bid decisions submitted by market participants.

[0169]

[0170] α={a1,a2,...a N} (40)

[0171] In the formula, a i This refers to the actions of the agent, which consist of the agent's participation in bidding and quantity decisions in the electricity market, carbon market, and green certificate market. These represent the bids made by agent i in the electricity market, carbon market, and green certificate market, respectively. These represent the reported volume of agent i in the electricity market, carbon market, and green certificate market, respectively.

[0172] (3) Rewards

[0173] The reward set R for the agent is the net profit obtained by each market participant in the electricity-carbon-green certificate market.

[0174] R = {R1, R2, ..., R} N} (41)

[0175] Thus, this method 200 constructs a two-layer optimization model for the coupled trading of electricity-carbon-green certificates by a power generation consortium. The upper-layer model represents the multi-market decision-making model for the consortium and various market participants, while the lower-layer model represents the trading clearing model for the electricity-carbon-green certificates market. The MADDPG algorithm is used to solve the two-layer model, obtaining the winning bid price, winning bid volume, and total revenue for each market participant in the electricity market, carbon market, and green certificate market, respectively. These results serve as the optimal decision-making scheme for the power generation consortium's participation in the electricity-carbon-green certificates market. This method 200 can promote the consumption of renewable energy and reduce electricity market clearing bias, thereby achieving a win-win situation for both renewable energy units and fossil fuel units.

[0176] In other embodiments, method 200 further includes: internally allocating the total revenue of the power generation consortium using the Shapely value method.

[0177] In a multi-market coupled environment, the distribution of benefits within a power generation alliance is a crucial step in optimizing resource allocation, safeguarding the interests of all members, and achieving fair cooperation. Since the alliance includes both renewable energy units and conventional thermal power units, whose revenue sources and contributions differ in the electricity market, carbon electricity market, and green certificate electricity market, a scientifically sound and reasonable revenue distribution method is necessary to ensure that all members actively participate and fully utilize their roles.

[0178] Payoff distribution methods based on cooperative game theory mainly include the nucleolus method, stable negotiation sets, and Shapley value method. These methods can effectively distribute the overall payoff of an alliance under different scenarios. Among them, the nucleolus method and stable negotiation sets require the construction of relatively complex mathematical models, and the calculation process is cumbersome. In contrast, the Shapley value method, with its simple calculation, reasonable results, and high fairness, has gained widespread acceptance in practical applications.

[0179] Given that the Shapley value method can effectively allocate revenue among alliance members, and possesses a clear mathematical foundation and high applicability, this embodiment chooses to use the Shapley value method to allocate the total revenue of the power generation alliance across multiple markets. In the Shapley value method, each member's allocated revenue is related not only to their direct contribution to the overall revenue of the alliance, but also to their marginal contribution under different joining orders within the alliance. By calculating the average marginal contribution of all members under various possible orderings, the Shapley value method ensures the fairness of revenue allocation and avoids unfair distribution caused by differences in revenue from a single market or a single member. The specific allocation method is as follows:

[0180]

[0181] In the formula: Let be the payoff of individual i in cooperative game alliance N; W be the number of composable sub-alliances in cooperative game alliance N; |s| and n be the number of individuals in sub-alliance S and alliance N respectively; υ(s) and υ(s-{i}) are the payoffs of sub-alliance before and after individual i joins the sub-alliance respectively.

[0182] To further illustrate the effectiveness of this method 200, a regional power system was selected as the research object for case analysis.

[0183] The two-layer coupled transaction model and its solution algorithm were analyzed using the IEEE-30 node standard test system. Four coal-fired power generators (G1-G4), two wind power generators (W1, W2), two photovoltaic power generators (S1, S2), and one power generation alliance (F1) were set up. The power generation alliance consists of two coal-fired power generators (FG1, FG2), one photovoltaic power generator (FS1), and one wind power generator (FW1). The parameters and access locations are shown in Table 1. The installed capacities of wind power generators W1, W2, and FW1 are 150MW, 100MW, and 150MW, respectively. The expected mean wind speed for wind power generation scenarios of wind generators W1 and W2 is 10m / s with a standard deviation of 3; the expected mean wind speed for wind power generation scenario of wind generator FW1 is 9m / s with a standard deviation of 2. The installed capacities of photovoltaic power generators S1, S2, and FS1 are 100MW, 150MW, and 150MW, respectively. The expected mean solar irradiance for photovoltaic power generators S1 and S2 is 0.45kW / m². 2 The standard deviation is 0.3, and the expected mean illuminance of the optical e-commerce FS1 is 0.4 kW / m². 2 The standard deviation is 0.28. Typical output curve for the load. Figure 5 As shown, the bidding range for market participants in the electricity market is set to [0, 400] yuan / MWh, the bidding range for the carbon market is [90, 110] yuan / t, and the range for the green certificate market is [25, 45] yuan / certificate.

[0184] Table 1 Unit Parameters

[0185]

[0186] In addition, the algorithm parameters were set as follows: maximum number of iterations per time period was 200; neural network parameters were updated every 5 iterations; the experience pool size was 1000, the discount factor γ was 0.9; the policy cross-entropy weight was 0.01; the actor network learning rate was 0.01, and the critic network parameters were the same as above. The simulation environment was: Intel(R) Xeon(R) Gold 5218R CPU@2.10GHz, RAM 128, the software configuration was Python 3.11, and the neural network was built using the PyTorch framework.

[0187] To deeply compare the strategic behavior changes and equilibrium results of the power generation alliance in the multi-market coupling of electricity, carbon trading, and green certificate markets, and to verify the effectiveness of the MADDPG algorithm, this study selects the traditional deep Q-network (DQN) algorithm for comparison with the MADDPG algorithm. Five different comparison schemes are set up, and the settings of each scenario are shown in Table 2.

[0188] Table 2 Scenario Setting

[0189]

[0190]

[0191] (1) Analysis of the market clearing results of electricity-carbon-green certificates

[0192] By analyzing the performance of the power generation consortium under different market participation scenarios, this study reveals that Scenario 5 demonstrates the best performance in employing the MADDPG algorithm and fully participating in the coupled trading of the electricity-carbon-green certificates market. The results are shown in Table 3. In Scenario 5, the power generation consortium achieved a total revenue of RMB 6.2214 million, significantly outperforming other scenarios.

[0193] In terms of market participation, Scenario 1, Scenario 2, and Scenario 5 all adopted the MADDPG algorithm, but their scope of market participation differed. Compared to Scenario 1 and Scenario 2, Scenario 1 only considered joint trading in the electricity-green certificate market, Scenario 2 only considered joint trading in the electricity-carbon market, while Scenario 5 encompassed the coupling of multiple markets including electricity, carbon, and green certificates. This diversified market participation brought a wider range of revenue sources to the power generation consortium. Specifically, the total revenue of Scenario 5 reached 6.2214 million yuan, significantly higher than the 4.8527 million yuan of Scenario 1 and the 6.0069 million yuan of Scenario 2, indicating that comprehensive participation in multiple markets can bring more substantial revenue to the power generation consortium.

[0194] From the perspective of algorithm performance, compared with scenarios three, four, and five, the MADDPG algorithm used in scenario five demonstrates its efficiency and advancement in handling multi-market coupled trading problems. Although scenario three uses the DQN algorithm, its total return of 5.7904 million yuan is lower than that of scenario five. Scenario four, which does not use deep reinforcement learning algorithms, has a total return of 4.0329 million yuan, significantly lower than that of scenario five. This significant difference not only proves the effectiveness of the MADDPG algorithm in optimizing market decisions but also highlights the important role of deep reinforcement learning algorithms in enhancing market competitiveness and economic benefits.

[0195] Table 3. Benefits of the Power Generation Consortium under Different Scenarios

[0196]

[0197] In conclusion, the effectiveness of Scenario 5 stems from the power generation consortium's comprehensive participation strategy in multi-market coupled trading and the efficient application of the MADDPG algorithm. This combination of strategy and algorithm not only enhances the consortium's returns across various markets but also achieves a balanced distribution of returns, helping the consortium maintain stable growth in complex market environments. The successful implementation of this scenario provides strong support for the consortium's strategy selection in multi-market coupled trading and offers valuable reference for other market participants, demonstrating the crucial role of the MADDPG algorithm in enhancing market competitiveness and economic efficiency.

[0198] (2) Algorithm performance and effectiveness analysis

[0199] Regarding algorithm performance, to verify the effectiveness of the MADDPG algorithm in supporting the decision-making of power generation alliances, this embodiment compares the DQN algorithm with the MADDPG algorithm, comparing their convergence speed, stability, and market returns during the training process at the same time. The curves showing the change in total market member returns with the number of iterations under different algorithms are shown below. Figure 6As shown in the figure, the MADDPG algorithm outperforms the DQN algorithm in electricity-carbon-green certificate market decision-making. Firstly, in terms of convergence speed, the MADDPG algorithm quickly stabilizes after approximately 2000 iterations, while the DQN algorithm requires more iterations to reach a similar stable state. Secondly, from a stability perspective, the MADDPG algorithm exhibits less fluctuation in returns after reaching a stable state, demonstrating higher stability, while the DQN algorithm still shows some fluctuations after stabilization. This may mean that the MADDPG algorithm can provide more reliable decision support in practical applications. Finally, from the perspective of total market returns, the MADDPG algorithm maintains a high return level throughout the iteration process, with a maximum return approaching 12 million yuan, while the maximum return of the DQN algorithm is slightly lower than 12 million yuan, and its overall return level is lower than that of the MADDPG algorithm. This indicates that the MADDPG algorithm is more effective in optimizing market decisions and can bring higher economic benefits to power generation users. In conclusion, the MADDPG algorithm demonstrates superiority in convergence speed, stability, and total return, making it a better choice for electricity-carbon-green certificate market decision-making.

[0200] The profit change curves of all market participants under the MADDPG solution algorithm are as follows: Figure 7 As shown, after 2000 iterations, the returns of each entity gradually stabilize, forming a market game equilibrium. In this equilibrium state, all market participants believe their bidding strategies are optimal and therefore do not change them, thus achieving market game equilibrium. The figure shows that, driven by the MADDPG algorithm, the returns of the nine power generation entities in the electricity-carbon-green certificate coupling market have all reached equilibrium. After a certain number of iterations, the return curves of each power generation entity show stability, indicating that they have adapted to market dynamics and developed effective bidding strategies. The multi-agent characteristic of the MADDPG algorithm allows market participants to interact strategically in an environment of incomplete information, effectively promoting the formation of market equilibrium. This equilibrium state not only provides a predictable return environment for power generation entities but also helps the electricity market achieve optimal resource allocation, ensuring the stability and economy of electricity supply. Therefore, the MADDPG algorithm is significantly effective for power generation alliances participating in the electricity-carbon-green certificate coupling market, not only optimizing individual returns but also promoting the rationalization of market resource allocation, providing strong support for the healthy development of the electricity market.

[0201] To better understand the bidding strategies and market performance of market players, and to assess the bidding strategies and market behavior of power generation consortia and other participants, this study analyzes the market bids, winning volumes, and clearing prices of each player. The findings reveal that: thermal power unit bids are relatively stable, possibly related to the relatively fixed nature of their generation costs; while power generation consortia' bids fluctuate significantly, reflecting their flexible strategies in the electricity-carbon-green certificate coupled market. This strategy allows consortia to adapt to market changes and optimize revenue. In contrast, the volatility of wind and solar power bids may be related to the intermittency and uncertainty of renewable energy. Power generation consortia's bids being higher than other market players at certain times may indicate a cost advantage during those periods or strategic bidding to gain a larger market share.

[0202] Furthermore, the power generation consortium achieved significant winning bids across multiple time periods, particularly during periods of higher electricity prices, indicating the success of its bidding strategy during these times. This success stems not only from the consortium's flexible bidding but also from the diversity and synergy of its resources. The consortium is able to dynamically adjust the output of thermal power and renewable energy based on market demand and price signals to maximize cost-effectiveness. While the winning bids for thermal power units remained relatively stable, electricity prices fluctuated significantly, likely due to changes in market demand and generation costs. Winning bids for wind and solar power varied considerably throughout the day, consistent with their generation characteristics, which increase during periods of favorable wind and sunshine conditions. Electricity price fluctuations also reflect changes in market supply and demand, with prices rising during peak demand periods and falling during off-peak periods.

[0203] An analysis of the results of the power generation alliance and other market participants' participation in the electricity-carbon-green certificate market clearing process reveals the unique advantages and strategic flexibility of the power generation alliance. In the electricity market, the alliance successfully adapted to market fluctuations by flexibly adjusting its bids, achieving significant winning bids, particularly during peak electricity price periods. This demonstrates not only its sensitivity to market demand but also its resource integration capabilities. The alliance also performed exceptionally well in the carbon and green certificate markets, with positive winning bids reflecting additional carbon allowances and green certificates obtained through internal renewable energy resources, all of which can be converted into economic benefits. This integration strategy allows the alliance to maximize cost-effectiveness not only in the electricity market but also to obtain additional economic incentives in the environmental market. This diversified market participation approach by the power generation alliance not only enhances its own market competitiveness but also provides strong support for the green transformation and sustainable development of the power industry, achieving the dual goals of economic benefits and environmental responsibility.

[0204] Furthermore, according to the method described in this application, the revenue of each entity within the power generation alliance is distributed, and the revenue comparison before and after joining the alliance is shown in Table 4. This demonstrates the necessity of forming a power generation alliance between thermal power units and new energy units. For thermal power units FG1 and FG2, their revenue increased significantly after joining the alliance. FG1's revenue increased from RMB 2.1023 million to RMB 2.1775 million, while FG2's revenue increased from RMB 2.1644 million to RMB 2.4886 million. This indicates that the alliance can improve the economic efficiency of thermal power units by optimizing resource allocation and market strategies. For new energy units FW1 and FS1, although they already had some revenue without the alliance, their revenue increased from RMB 796,900 to RMB 933,200 and from RMB 361,000 to RMB 622,100, respectively, after joining the alliance. This shows that the alliance can provide a more stable market environment and more effective revenue protection for new energy units.

[0205] Table 4 Comparison of Power Generation Alliance Benefits Before and After Distribution

[0206]

[0207] In summary, the formation of a power generation alliance between thermal power units and renewable energy units can not only improve their respective economic efficiency but also enhance the stability and flexibility of the power grid by optimizing output and revenue distribution. The alliance enables each unit to better respond to market changes and improves overall market competitiveness. Therefore, establishing a power generation alliance is a mutually beneficial strategy for both thermal power units and renewable energy units, contributing to the healthy development of the electricity market.

[0208] In summary, by deeply analyzing the bidding strategies and revenue distribution of power generation consortia under different market environments, this application proposes a two-layer coupled trading model combining a deep reinforcement learning algorithm (MADDPG). The upper-layer model is responsible for the market decisions of the power generation consortium, while the lower-layer model is responsible for the clearing of transactions in the electricity-carbon-green certificate market. Numerical examples demonstrate that this model can effectively improve the total revenue of power generation consortia in the electricity-carbon-green certificate market and achieve a balanced distribution of revenue.

[0209] The main conclusions are as follows:

[0210] (1) The two-layer coupled trading model based on the MADDPG algorithm constructed in this application can effectively improve the total revenue of the power generation alliance in the electricity-carbon-green certificate market and achieve a balanced distribution of revenue. This model optimizes the bidding strategy of the power generation alliance through upper-level market decision-making and lower-level market clearing mechanisms, enabling it to obtain more economic benefits in a highly competitive market. This not only promotes the consumption of renewable energy but also reduces electricity market clearing bias, achieving a win-win situation for both renewable energy units and fossil fuel units.

[0211] (2) Forming a power generation alliance can bring significant economic benefits to both thermal power units and new energy units. By integrating units of different energy types, the alliance optimizes resource allocation and market strategies, improving the economic efficiency of thermal power units, while providing a more stable market environment and more effective revenue guarantees for new energy units. This cooperative model enhances the stability and flexibility of the power grid, improves overall market competitiveness, and is a mutually beneficial strategy.

[0212] Figure 8 A block diagram of the physical components (i.e., hardware) of a computing device 800 according to some embodiments of this application is shown. In a basic configuration, the computing device 800 includes at least one processing unit 802 and a system memory 804. According to one aspect, depending on the configuration and type of the computing device, the processing unit 802 may be implemented as a processor. The system memory 804 includes, but is not limited to, volatile memory (e.g., random access memory), non-volatile memory (e.g., read-only memory), flash memory, or any combination of such memories. According to one aspect, the system memory 804 includes an operating system 805 and a program module 806, the program module 806 including computer programs / instructions for performing various methods that can instruct the processing unit 802 to perform various methods. In some embodiments of this application, the program module 806 stores computer programs / instructions for performing method 200 according to this application.

[0213] According to one aspect, operating system 805 is, for example, suitable for controlling the operation of computing device 800. Furthermore, the example is practiced in conjunction with graphics libraries, other operating systems, or any other applications, and is not limited to any particular application or system. Figure 8 The basic configuration is illustrated by the components within the dashed lines 808. According to one aspect, the computing device 800 has additional features or functions. For example, according to one aspect, the computing device 800 includes additional data storage devices (removable and / or non-removable), such as disks, optical discs, or magnetic tapes. This additional storage... Figure 8 The image is shown by removable storage 809 and non-removable storage 810.

[0214] As stated above, according to one aspect, a program module 806 is stored in system memory 804. According to one aspect, program module 806 can be implemented as one or more computer program products. This application does not limit the type of computer program product, and may include, for example, email, word processing applications, spreadsheet applications, database applications, slideshow applications, drawing or computer-aided applications, web browsers, etc. In some embodiments according to this application, computer programs / instructions related to method 200 are packaged into a computer program product that, when executed by a processor (i.e., processing unit 802), implements method 200 according to this application.

[0215] According to one aspect, examples can be practiced on circuits including discrete electronic components, packaged or integrated electronic chips containing logic gates, circuits utilizing microprocessors, or on a single chip containing electronic components or a microprocessor. For example, it can be practiced via wherein... Figure 8 Each or many of the components shown can be implemented as an example of a system-on-a-chip (SOC) integrated on a single integrated circuit. According to one aspect, such an SOC device may include one or more processing units, graphics units, communication units, system virtualization units, and various application functions, all integrated (or “burned in”) as a single integrated circuit onto a chip substrate. When operated via the SOC, the functions described herein can be operated via dedicated logic integrated on a single integrated circuit (chip) with other components of the computing device 800. Embodiments of this application can also be implemented using other techniques capable of performing logical operations (e.g., AND, OR, and NOT), including but not limited to mechanical, optical, fluid, and quantum technologies. Additionally, embodiments of this application can be implemented within a general-purpose computer or in any other circuit or system.

[0216] According to one aspect, the computing device 800 may also have one or more input devices 812, such as a keyboard, mouse, pen, voice input device, touch input device, VR motion capture input device, etc. It may also include output devices 814, such as a display, speaker, printer, etc. The foregoing devices are examples and other devices may also be used. The computing device 800 may include one or more communication connections 816 that allow communication with other computing devices 818. Examples of suitable communication connections 816 include, but are not limited to: RF transmitter, receiver and / or transceiver circuitry; Universal Serial Bus (USB), parallel and / or serial ports.

[0217] As used herein, the term computer-readable medium includes computer storage medium. Computer storage medium can include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information (e.g., computer-readable instructions, data structures, or program modules). System memory 804, removable storage 809, and non-removable storage 810 are examples of computer storage media (i.e., memory storage). Computer storage media can include random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape, magnetic tape, disk storage or other magnetic storage devices, or any other article of manufacture that can be used to store information and is accessible by computing device 800. According to one aspect, any such computer storage medium can be part of computing device 800. Computer storage media does not include carrier waves or other propagated data signals.

[0218] According to one aspect, a communication medium is implemented by computer-readable instructions, data structures, program modules, or other data in a modulated data signal (e.g., a carrier wave or other transmission mechanism), and includes any information transmission medium. According to one aspect, the term "modulated data signal" describes a signal having one or more sets of characteristics or altered in a manner that encodes information in the signal. By way of example and not limitation, a communication medium includes wired media such as wired networks or direct wired connections, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0219] Optionally, in this method, generating multiple preset wind and solar power output scenarios and calculating the probability of each scenario occurring includes: using a probability distribution model to generate power output curves for wind speed and light intensity respectively; generating a first number of preset wind and solar power output scenarios based on the wind speed power output curve and the light intensity power output curve; and calculating the probability of each scenario occurring based on the relationship between wind speed, light intensity and actual wind and solar power output.

[0220] Optionally, in this method, constructing a day-ahead clearing model for the electricity market also includes: using the SCUC model and SCED model to clear the market with the goal of minimizing electricity purchase costs, and obtaining the unit start-up and shutdown plan, the unit day-ahead output, and the day-ahead node price.

[0221] Optionally, in this method, the day-ahead clearing model of the electricity market, the clearing model of the carbon market, and the clearing model of the green certificate market all include multiple constraints.

[0222] Optionally, in this method, the MADDPG algorithm is used to solve the two-layer coupled trading model, including: using each market participant as an agent of the MADDPG algorithm, using the electricity-carbon-green certificate market as the environment of the MADDPG algorithm, repeatedly iterating the decision model and the coupled clearing model until the optimal action selection strategy is obtained, wherein the optimal action selection strategy includes the winning bid result and the corresponding bid decisions submitted by each market participant.

[0223] Optionally, this method also includes: internally allocating the total revenue of the power generation alliance using the Shapely value method.

[0224] Optionally, this application also includes a computing device comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing the methods described above.

[0225] Optionally, this application also includes a computer-readable storage medium storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform the method as described above.

[0226] Optionally, this application also includes a computer program product comprising a computer program / instructions, wherein the computer program / instructions, when executed by a processor, implement the method as described above.

[0227] The various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the methods and apparatus of this application, or certain aspects or portions thereof, can take the form of program code (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, USB flash drive, floppy disk, CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into and executed by a machine such as a computer, the machine becomes an apparatus for practicing this application.

[0228] When the program code is executed on a programmable computer, the computing device generally includes a processor, a processor-readable storage medium (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store program code; the processor is configured to execute the method of this application according to instructions in the program code stored in the memory.

[0229] By way of example, and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer-readable instructions, data structures, program modules, or other data. Communication media generally embodies computer-readable instructions, data structures, program modules, or other data in the form of modulated data signals such as carrier waves or other transmission mechanisms, and includes any information delivery medium. Any combination of the above is also included within the scope of readable media.

[0230] In the specification provided herein, the algorithms and displays are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used with the examples of this application. Based on the above description, the required structure for constructing such a system is obvious. Furthermore, this application is not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing preferred embodiments of this application.

[0231] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0232] As used herein, unless otherwise specified, the use of ordinal numbers such as “first,” “second,” “third,” etc., to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, sequence, or any other manner. Furthermore, the quantifier “multiple” means “two” and / or “more than two.”

[0233] Although this application has been described with reference to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of this application described herein. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and edibility purposes, and not for the purpose of interpreting or limiting the subject matter of this application. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of this application. The disclosure of this application is illustrative and not restrictive in relation to its scope.

Claims

1. A decision-making method for power generation consortia participating in the electricity-carbon-green certificate market based on the MADDPG algorithm, wherein the power generation consortia include renewable energy units and conventional thermal power units, and the market also includes traditional thermal power generation entities and wind and solar new energy power generation entities, the method comprising: Based on the sales revenue of each market participant in the electricity-carbon-green certificate market, a bidding decision model for each market participant in the electricity-carbon-green certificate market is constructed as a decision model. This decision model includes: a bidding decision model for the power generation alliance based on its revenue and generation costs in the electricity market, carbon market, and green certificate market; a bidding decision model for traditional thermal power generation entities based on their revenue and generation costs in the electricity market and carbon market; and a bidding decision model for wind and solar renewable energy power generation entities based on their revenue in the electricity market, carbon market, and green certificate market. With the objectives of minimizing electricity purchase costs in the electricity market, maximizing welfare in the carbon market, and maximizing welfare in the green certificate market, respectively, a day-ahead clearing model for the electricity market, a clearing model for the carbon market, and a clearing model for the green certificate market are constructed as coupled clearing models. A two-layer coupled transaction model is constructed using the decision model as the upper-layer model and the coupled clearing model as the lower-layer model. The MADDPG algorithm is used to solve the two-layer coupled trading model to determine the winning bid price, winning bid volume, and total revenue of each market participant in the electricity market, carbon market, and green certificate market. The construction of the bidding decision model for the power generation alliance includes: using the objective function, quantity reporting constraints, and bidding constraints of the power generation alliance to construct the bidding decision model for the power generation alliance; The reporting constraints of the power generation alliance include: in, This indicates the amount of electricity generated by the power generation alliance reported in the day-ahead energy market at time t; and This represents the minimum and maximum amount of electricity won by the power generation consortium in the day-ahead energy market at time t; For the carbon emissions of the power generation alliance; This represents the amount of electricity generation consortiums have won bids in the carbon market; The total carbon allowance for the power generation consortium; This represents the amount of green certificates won by the power generation consortium in the green certificate market. This indicates the number of green certificates obtained by the power generation consortium.

2. The method as described in claim 1, wherein, The bidding decision model for the power generation consortium includes: Taking into account the uncertainties of wind and solar power output, multiple preset wind and solar power output scenarios are generated and the probability of each scenario occurring is calculated. Based on various wind and solar power output scenarios, and considering the revenue and power generation costs of the power generation alliance in the electricity market, carbon market, and green certificate market, the objective function is to maximize the total revenue of the power generation alliance. Based on the power generation alliance's constraints on electricity volume declaration in the electricity market, carbon quota declaration in the carbon market, and green certificate declaration in the green certificate market, the declaration quantity constraints for the power generation alliance are generated. Generate the bidding constraints for the power generation alliance in the electricity market, carbon market, and green certificate market; Using the objective function of the power generation alliance and the reported quantity constraints and the quoted price constraints, a pricing decision model for the power generation alliance is constructed.

3. The method as described in claim 2, wherein, Determining the total revenue of the power generation consortium includes: Under various wind and solar power output scenarios, the electricity market revenue is determined based on the day-ahead energy market clearing price and the electricity volume won by the power generation alliance in the day-ahead energy market. In various wind and solar power output scenarios, carbon market revenue is calculated based on the carbon market clearing price and the amount of carbon market bids won by the power generation alliance. In various wind and solar power output scenarios, the revenue from the green certificate market is calculated based on the green certificate market clearing price and the number of green certificate contracts won by the power generation alliance in the green certificate market. The total revenue of the power generation consortium is obtained by subtracting the power generation costs of conventional thermal power units in the consortium from the sum of the revenue from the electricity market, the carbon market, and the green certificate market.

4. The method as described in claim 2 or 3, wherein, The carbon allowance declaration constraints include: the amount of carbon allowances won by the power generation consortium in the carbon market shall not be less than the carbon emissions of conventional thermal power units within the consortium, and shall not exceed the total carbon allowances obtained by the consortium. The total carbon allowances obtained by the power generation consortium include: carbon allowances obtained by conventional thermal power units in the power generation consortium and carbon allowances obtained by renewable energy units in the power generation consortium through certified voluntary emission reductions.

5. The method as described in claim 2 or 3, wherein, The constraints on the number of green certificate applications include: the number of green certificate contracts won by the power generation consortium in the green certificate market shall not exceed the number of green certificates obtained by the power generation consortium.

6. The method as described in claim 2 or 3, wherein, The objective function of the power generation alliance is expressed as: in, In the formula, F GA This represents the total revenue of the power generation consortium; This indicates the power generation alliance's earnings in the day-ahead market; This indicates the benefits that the power generation alliance receives in the carbon market; This indicates the power generation consortium's revenue in the green certificate market; π represents the power generation cost of the h-th conventional thermal power unit in the power generation consortium; H represents the number of conventional thermal power units in the power generation consortium; s The probability of scenario s, which contributes to the wind and solar power, occurring; S represents the number of such scenarios. This represents the energy market clearing price at time t. λ represents the amount of electricity won by the power generation consortium in the day-ahead energy market at time t; ca To clear prices in the carbon market; The amount of electricity generation alliances won in the carbon market; λ TGC Clearing out prices in the green certificate market; The amount of green certificates won by the power generation consortium in the green certificate market; a h b h c h For thermal power units, T = 24, t = 1, 2, ..., 24; Let t represent the power generation of conventional thermal power units in the power generation alliance at time t.

7. The method of claim 4, wherein, The carbon emissions of conventional thermal power units in the aforementioned power generation consortium are expressed as follows: in, For the carbon emissions of the power generation consortium; π s The probability of scenario s occurring as a landscape power-generating scene; S represents the number of landscape power-generating scenes; κ GH Carbon emission coefficient; Let t be the power generation of conventional thermal power units in the power generation alliance at time t; H is the number of conventional thermal power units in the power generation alliance; T = 24, t = 1, 2, ..., 24.

8. The method of claim 5, wherein, The number of green certificates obtained by the power generation consortium is expressed as follows: in, Indicates the number of green certificates obtained by the power generation consortium; π s The probability of scenario s occurring as a landscape power-generating scene; S represents the number of landscape power-generating scenes; κ TGC For new energy green certificates and electricity conversion coefficient; Let t represent the power generation of renewable energy generators in the power generation alliance at time t; E represents the number of renewable energy generators in the power generation alliance; a represents the percentage of renewable energy generators in the power generation alliance that are converted into certified voluntary emission reductions; T = 24, t = 1, 2, ..., 24.

9. The method as described in claim 2 or 3, wherein, The process of generating multiple preset wind and solar power output scenarios and calculating the probability of each scenario occurring includes: Using a probability distribution model, output curves for wind speed and light intensity are generated respectively; Based on the wind speed output curve and the light intensity output curve, a first number of preset wind and solar power output scenarios are generated, and the probability of each wind and solar power output scenario is calculated according to the relationship between wind speed, light intensity and actual wind and solar power output.

10. The method according to any one of claims 1-3, wherein, The construction of a day-ahead clearing model for the electricity market also includes: With the goal of minimizing electricity purchase costs, the SCUC and SCED models are used for market clearing to obtain unit start-up and shutdown plans, day-ahead unit output, and day-ahead node electricity prices.

11. The method according to any one of claims 1-3, wherein, The day-ahead clearing model for the electricity market, the carbon market clearing model, and the green certificate market clearing model all include multiple constraints.

12. The method according to any one of claims 1-3, wherein, The MADDPG algorithm is used to solve the two-layer coupled trading model, including: Using each market participant as an agent for the MADDPG algorithm, and the electricity-carbon-green certificate market as the environment for the MADDPG algorithm, the decision model and the coupled clearing model are iterated repeatedly until the optimal action selection strategy is obtained. The optimal action selection strategy includes the bidding results and the corresponding bid decisions submitted by each market participant.

13. The method according to any one of claims 1-3, further comprising: The total revenue of the power generation consortium is internally allocated using the Shapely value method.

14. A computing device, comprising: One or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing the method as described in any one of claims 1-13.

15. A computer-readable storage medium storing one or more programs, said one or more programs including instructions that, when executed by a computing device, cause the computing device to perform the method as claimed in any one of claims 1-13.

16. A computer program product comprising a computer program / instructions, wherein, When the computer program / instructions are executed by the processor, they implement the method of any one of claims 1-13.

Images