A decision-making optimization method for power generators considering the interaction mechanism of electricity and carbon coupling

Abstract

A decision-making optimization method for power generators considering an electric-carbon coupling interaction mechanism includes: considering the power generation loss, declaration curve form and other characteristics of the power generator, and constructing a decision model for participating in the electric-carbon coupling interaction; constructing an electric-power spot mechanism clearing model based on the electric-power spot mechanism rules and the declaration information of the power generator; based on the electric-power spot mechanism clearing results, allocating free carbon quotas according to the generating power of each power generator's unit, and constructing a carbon quota allocation model; based on the electric-power spot mechanism clearing results and the carbon quota allocation results, combined with the generating carbon emission intensity of each power generator's unit, constructing a carbon emission mechanism clearing model; using the DDPG deep reinforcement learning method to solve the optimal strategy of the power generator under the electric-carbon coupling mechanism. The present invention can solve the optimal strategy of the power generator under the electric-carbon coupling interaction mechanism with incomplete information, and at the same time can effectively reduce the carbon emissions of power generation in the power system and improve the clean and low-carbon level of the power system.

Description

Technical Field

[0001] The present invention relates to the technical field of electric power regulation and control, and in particular to a decision-making optimization method for power generators taking into account an electric-carbon coupling interaction mechanism. Background Art

[0002] As a basic industry of the national economy, the power industry occupies a dominant position in the total carbon emissions, and its low-carbon transformation plays a vital role in promoting the in-depth implementation of the "dual carbon" goals. With the gradual advancement of the carbon emission interaction mechanism, the carbon emission interaction mechanism will play an increasingly important role in promoting energy conservation, emission reduction, and green and low-carbon development in the power industry. With the development of the carbon emission interaction mechanism, the carbon emission interaction process will have a greater impact on the clearing results of the power spot mechanism and the decision-making of power generators. The power spot mechanism and the carbon emission interaction mechanism influence each other through coupling, and jointly determine the decision-making behavior of the power generator. The effective synergy between the two will be conducive to forming a joint force to jointly promote the development of clean energy and carbon emission reduction in the power industry.

[0003] In the process of pursuing carbon emission reduction in the power system, it is particularly important to build a power market system that integrates with the carbon emission interaction mechanism. With the gradual improvement and perfection of the carbon emission interaction mechanism, its role in promoting energy conservation and emission reduction in the power industry and promoting green and low-carbon development has become increasingly prominent. At the same time, the close coupling between the carbon emission interaction mechanism and the electricity spot mechanism has a profound impact on the decision-making behavior of power generators. The effective synergy of the two can not only stimulate the development potential of clean energy, but also significantly promote the carbon emission reduction process in the power industry.

[0004] However, existing decision analysis methods for power producers are unable to cope with the high coupling of power interaction mechanisms and carbon emission interaction mechanisms in the context of new power system construction. Traditional optimization analysis methods are difficult to fully consider the non-convex characteristics of generator set costs and operating characteristics during the modeling process, and the number of complementary relaxation conditions generated during the linearization of the model is huge, which leads to severe challenges for this method in dealing with power producer decision problems under large-scale systems and complex mechanisms. In addition, traditional models are only applicable to complete information game scenarios, while the information environment in the actual power spot market is often limited, which makes it difficult for the solution results of traditional models to truly reflect the actual decision-making behavior of power producers.

[0005] Therefore, the field urgently needs to find a decision-making optimization method for power producers that considers the coupling of the electricity spot mechanism and the carbon emission rights interaction mechanism. Currently, no description or report of similar technology to the present invention has been found, and similar information at home and abroad has not been collected. Summary of the invention

[0006] In view of the above-mentioned deficiencies in the prior art, the present invention provides a decision optimization method for power generators taking into account the electricity-carbon coupling interaction mechanism, and also provides a corresponding terminal and medium.

[0007] To achieve the above purpose, the technical solution of the present invention is as follows:

[0008] A power generator decision optimization method considering the electricity-carbon coupling interaction mechanism is characterized in that it includes the following steps:

[0009] S1. Construct a decision-making model that takes into account the participation of power generators in the electricity-carbon coupling interaction, determine the objective function and optimal operation constraints of the power generators' participation in the electricity-carbon coupling interaction mechanism, the objective function of the power generators' participation in the electricity-carbon coupling interaction mechanism takes into account the incentives for power generators to sell electricity in the electricity spot market, the incentives for selling carbon quotas in the carbon emission market, and power generation losses, the optimal operation constraints include upper and lower limits on power generation losses to ensure that the power generators' reported losses are within a reasonable range, and monotonicity constraints on the power generation loss curve to confirm that the loss curve reported by the power generators conforms to the actual operating rules;

[0010] S2. Construct a power spot mechanism clearing model, determine the objective function and optimal operation constraints of the power spot mechanism clearing model, wherein the objective function of the power spot mechanism clearing model is to minimize the total power generation loss of the system, and the optimal operation constraints include node power balance constraints, upper and lower power constraints of generator sets, climbing constraints of generator sets, upper and lower power constraints of branch currents, node voltage phase angle constraints, and balanced node voltage phase angle constraints;

[0011] S3. Based on the clearing results of the electricity spot mechanism, free carbon quotas are allocated according to the generating capacity of each power producer’s units, and a carbon quota allocation model is constructed;

[0012] S4. Based on the clearing results of the electricity spot mechanism and the carbon quota allocation results, combined with the carbon emission intensity of each power company's units, a step-by-step carbon price is used to construct a carbon emission mechanism clearing model;

[0013] S5. Use the DDPG deep reinforcement learning method to solve the decision-making model of power generators under the electricity-carbon coupling mechanism and seek the optimal strategy.

[0014] Furthermore, in step S1, the objective function of the power generator participating in the electricity-carbon coupling interaction mechanism includes the total incentive of the power generator, which is composed of the power generator's incentive for selling electricity through the electricity spot interaction mechanism, the incentive for selling carbon quotas through the carbon emission interaction mechanism, and the power generation loss through the electricity spot interaction mechanism, and the formula is as follows: Where: Provide total incentives for generators; Provide incentives for power generators to participate in the electricity spot exchange mechanism and sell electricity; Incentives for power generators to participate in the carbon emission exchange mechanism and sell carbon quotas; The power generation loss of the power generator participating in the power spot interactive mechanism. Further, in step S2, the objective function of the power spot mechanism clearing model is as follows: Where: Reporting for generators Time period generator No. The loss of power generation capacity, for Time period generator No. The output of the power generation capacity of the segment.

[0015] Further, the step S3 constructs a carbon quota allocation model, specifically including: S31. Calculate the total amount of carbon quota allowed to be emitted , which is related to the total power generation of the generator set, as follows: Where: is the carbon quota benchmark coefficient, for Time period generator No. Output of the generating capacity of the segment;

[0016] S32. Allocate free carbon quotas to each power generator based on the clearing results of the electricity spot mechanism , the formula is as follows: Where: for Time period generator The free carbon quota allocation coefficient.

[0017] Furthermore, the step S4 constructs a carbon emission mechanism clearing model, which specifically includes:

[0018] S41: Calculate the actual carbon emissions of power generation by the power generator using the following formula: In the formula, for Time period generator Actual carbon emissions from electricity generation, For generator sets Carbon emission intensity per unit of electricity generation;

[0019] S42: Calculate the carbon quota interaction of the power generator, that is, the difference between the actual carbon emissions and the free carbon quota, the formula is as follows: In the formula, yes Time period generator Carbon quotas participating in carbon quota interactions;

[0020] S43: Set a step-by-step carbon quota coefficient, and set different carbon quota coefficients for different carbon quota interaction amounts. The formula is as follows:

[0021] S44: Calculate the incentive for power generators to sell carbon quotas using the following formula: In the formula, for Time period generator Incentives to participate in carbon quota interactions.

[0022] Further, the step S5 is solved by using the DDPG deep reinforcement learning method, which specifically includes:

[0023] S51: Establish a reinforcement learning environment and agent information, taking the power generator as the agent and the clearing process of the electricity-carbon coupling interaction mechanism as the environment;

[0024] S52: Establish the Markov decision process of DDPG reinforcement learning algorithm to describe the dynamic process of interaction between the agent and the environment;

[0025] S53: Establish the state space of the power generator agent: The formula includes the reporting period for power generators to participate in the electricity-carbon coupling interaction mechanism. , Load demand during the period , Time-of-day power generators Report the power generation losses of each power generation capacity segment , The remaining power generators in the period report the power generation losses of each power generation capacity segment ;

[0026] S54: Establish the action space of the power producer agent: In the formula, including Time-of-day power generators Declare the losses of each generation capacity segment;

[0027] S55: Establish a reward function to motivate the agent to adopt the optimal strategy;

[0028] S56: Establish a cumulative discounted reward function and an action value function to evaluate the quality of the agent's strategy;

[0029] S57: Establish the policy network and value network respectively, and update the network parameters by gradient ascent and gradient descent methods;

[0030] S58: Establish a solution process for the power generator decision optimization method based on the DDPG algorithm, and finally output the optimal strategy for each intelligent agent through multiple iterative training.

[0031] Further, the step S58: establishing a power producer decision optimization method based on the DDPG algorithm, comprising:

[0032] (1) Define the number of training times , Discount Rate , policy network learning rate , value network learning rate , Value network update frequency ,Soft update frequency ;

[0033] (2) Initialize the policy network parameters of each agent , Value Network Parameters , target strategy network parameters and target value network parameters ;

[0034] (3) The agent obtains the current state of the electric-carbon coupling interaction mechanism ;

[0035] (4) The agent makes decisions based on the current state and outputs actions ;

[0036] (5) The electricity-carbon coupling interaction mechanism is cleared and rewards are fed back to each intelligent agent ; and output the state of the next moment ;

[0037] (6) In the experience replay pool Storing quads in ;

[0038] (7) If the current training step number Divisibility , from the experience replay pool Medium batch sampling quad , calculate the target strategy network output action of each agent and target value network output evaluation , update the value network parameters ;

[0039] (8) If the current training step number Divisibility , update the policy network parameters , soft update target value network parameters and target policy network parameters ;

[0040] (9) Repeat steps (3) to (8) until the number of training times reaches the set value. ;

[0041] (10) Output the optimal strategy of each agent after training.

[0042] Due to the adoption of the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art:

[0043] The present invention provides a decision-making optimization method for power generators that takes into account the electricity-carbon coupling interaction mechanism. It combines the electricity spot mechanism and the carbon emission mechanism to form an electricity-carbon coupling optimization system, introduces carbon quota allocation and interaction mechanism, and realizes the coupling between electricity-carbon mechanisms by purchasing carbon quotas to offset carbon emissions, thereby reducing the total carbon emissions of system power generation and promoting the cleanness of the power industry and the realization of carbon emission reduction goals.

[0044] The optimization method of the power generator decision optimization method considering the electric-carbon coupling interaction mechanism provided by the present invention can help power generators formulate their optimal strategies, and at the same time provide quantitative reference and evaluation basis for the evolutionary analysis of different mechanism designs, helping to compare the advantages and disadvantages of mechanism designs and judge their effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0046] Figure 1 The present invention is a flowchart of a decision-making optimization method for power generators that takes into account the electricity-carbon coupling interaction mechanism in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0047] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings, but this should not limit the protection scope of the present invention.

[0048] The method of the present invention comprehensively considers the dual roles of power generators in the electricity spot market and the carbon emission market, and helps power generators maximize their economic benefits while meeting electricity demand and carbon emission restrictions by constructing a series of models and optimization algorithms.

[0049] See also Figure 1 , Figure 1 This is a workflow diagram of a power producer decision optimization method considering the electricity-carbon coupling interaction mechanism in a preferred embodiment of the present invention, as shown in the figure, including the following steps:

[0050] Step S1: Construct a decision model for power generators to participate in electricity-carbon coupling interaction

[0051] S11: Determine the objective function, which comprehensively considers the incentives for power generators to sell electricity in the electricity spot market, the incentives for selling carbon quotas in the carbon emission market, and power generation losses.

[0052] S12: Set upper and lower limits for power generation losses to ensure that the losses reported by power generators are within a reasonable range.

[0053] S13: Set the monotonicity constraint of the power generation loss curve to ensure that the loss curve reported by the power generator conforms to the actual operating rules.

[0054] Step S2: Constructing a clearing model for the electricity spot mechanism

[0055] S21: Determine the objective function to minimize the total power generation loss of the system.

[0056] S22: Set a series of constraints, including node power balance, upper and lower limits of generator power, generator ramping, upper and lower limits of branch power flow, node voltage phase angle, and balanced node voltage phase angle, to ensure the stable operation of the power system.

[0057] Step S3: Constructing a carbon quota allocation model

[0058] S31: Calculate the total amount of carbon quotas allowed to be emitted, which is related to the total power generation of the power generation units.

[0059] S32: Allocate free carbon quota to each power generation unit based on the clearing results of the electricity spot mechanism.

[0060] Step S4: Construct a carbon emission mechanism clearing model

[0061] S41: Calculate the actual carbon emissions of electricity generation by power generators.

[0062] S42: Calculate the carbon quota interaction of the power generator, that is, the difference between the actual carbon emissions and the free carbon quota.

[0063] S43: Setting a stepped carbon quota coefficient, and setting different coefficients for different carbon quota interaction amounts.

[0064] S44: Calculate the incentives for generators to sell carbon allowances.

[0065] Step S5: Solve using DDPG deep reinforcement learning method

[0066] Embodiment: A power generator decision optimization method that comprehensively considers the interaction mechanism of power generation loss, power spot market and carbon emission market. The method comprises the following steps:

[0067] Step S1: Considering the power generation loss, declaration curve form and other characteristics of the power generator, a decision model for its participation in the electricity-carbon coupling interaction is constructed to determine the objective function and optimization operation constraints;

[0068] Step S11: Considering the power generation loss of the power generator, the objective function of the power generator participating in the electricity-carbon coupling interaction mechanism is determined as:

[0069] (1)

[0070] Where: Provide total incentives for generators; Provide incentives for power generators to participate in the electricity spot exchange mechanism and sell electricity; Incentives for power generators to participate in the carbon emission exchange mechanism and sell carbon quotas; To reduce power generation losses for power generators participating in the electricity spot interaction mechanism.

[0071] The calculation formula is as follows: (2)

[0072] in, is the node number, Number the generator set. is the generation capacity segment number, is the time period number, is the total power generation capacity segment number, is the total number of time periods, for Time period node The corresponding electricity sales index is for Time period generator No. The output of the power generation capacity of the segment, is a fixed time interval.

[0073] The calculation formula is as follows:

[0074] (3)

[0075] in, for Time period generator The carbon quota interaction index, for Time period generator of the free carbon quotas allocated, for Time period generator The carbon emissions from actual electricity generation.

[0076] The calculation formula is as follows:

[0077] (4)

[0078] in, , , For generator sets The power generation loss curve coefficient is in the form of a quadratic function, for Time period generator total output.

[0079] Step S12: Setting upper and lower limits of power generation loss:

[0080] (5)

[0081] in, Reporting for generators Time period generator No. The loss of power generation capacity, for Time period generator No. Minimum loss of power generation capacity , for Time period generator No. Minimum loss of power generation capacity in this section.

[0082] Step S13: Setting the monotonicity constraint of the power generation loss curve:

[0083] (6)

[0084] This constraint reflects that the power generation loss curve reported by the power generator should be in a monotonically increasing form. The loss of power generation capacity in the first section should not be less than Unit loss of generating capacity of the segment.

[0085] Step S2: Based on the electricity spot mechanism rules and the information reported by the power generator, a clearing model of the electricity spot mechanism is constructed to determine the objective function and the optimization operation constraints; the specific implementation method is as follows:

[0086] Step S21: Considering the total power generation loss of the power system, the objective function of the power spot mechanism clearing model is determined as:

[0087] (7)

[0088] The goal is to minimize the total power generation losses in the system.

[0089] Step S22: Setting node power balance constraints:

[0090] (8)

[0091] in, is the load number, For Node The generator set on For Node The load set on For the node A collection of connected nodes, for Time load The power demand, To connect nodes and nodes The line admittance, for Time Node The voltage phase angle, for Time Node The voltage phase angle.

[0092] Step S23: Set the upper and lower limits of the power of the generator set:

[0093] (9)

[0094] in, For generator sets No. The lower power limit of the power generation capacity of the segment, For generator sets No. The power upper limit of the power generation capacity of the segment.

[0095] Step S24, setting the generator set climbing constraint:

[0096] (10)

[0097] in, For generator sets The lower climbing limit of For generator sets The upper limit of the ramp, the ramp refers to the power difference of the generator set in adjacent time periods.

[0098] Step S25: Set the upper and lower limit constraints of the branch power flow:

[0099] (11)

[0100] in, Refers to the connection node and nodes The maximum transmission power of the line.

[0101] Step S26, setting node voltage phase angle constraints:

[0102] (12)

[0103] in, For Node The minimum value of the voltage phase angle, For Node The maximum value of the voltage phase angle.

[0104] Step S27, setting the voltage phase angle constraint of the balancing node:

[0105] (13)

[0106] in, : is the voltage phase angle value of the balance node, and the number of the balance node is set to 1.

[0107] Step S3: Based on the clearing results of the electricity spot mechanism, free carbon quotas are allocated according to the generating power of each power producer, and a carbon quota allocation model is constructed; the specific implementation method is as follows:

[0108] Step S31: The total amount of carbon quota allowed to be emitted depends on the total power generation of the generator set on the operating day. The calculation formula is as follows:

[0109] (14)

[0110] in, is the total amount of carbon quota, is the carbon quota benchmark coefficient.

[0111] Step S32: The carbon quota allocation of the generator set depends on the actual power generation of each unit in the clearing result of the power spot mechanism. The calculation formula is as follows:

[0112] (15)

[0113] (16)

[0114] in, yes Time period generator of free carbon quotas, for Time period generator The free carbon quota allocation coefficient.

[0115] Step S4: Based on the clearing results of the electricity spot mechanism and the carbon quota allocation results, combined with the carbon emission intensity of each power generator’s unit power generation, a carbon emission mechanism clearing model is constructed in the form of a stepped carbon price. The specific implementation method is as follows:

[0116] Step S41: The carbon emissions of the power generators depend on the actual power generation of the power generating units and the unit carbon emission intensity of the power generating units in the clearing results of the power spot mechanism. The calculation formula is as follows:

[0117] (17)

[0118] in, for Time period generator Actual carbon emissions from electricity generation, For generator sets The carbon emission intensity per unit of electricity generation.

[0119] Step S42: The amount of carbon quota interaction of the power generator depends on the actual carbon emissions of the power generation of the power generator and the free carbon quota. The calculation formula is as follows:

[0120] (18)

[0121] in, yes Time period generator The carbon quota participating in the carbon quota interaction is the difference between the actual carbon emissions from power generation and the free carbon quota.

[0122] Step S43: Step carbon quota coefficient: divide the carbon quota into multiple intervals, set carbon quota coefficients of different sizes for different carbon quota interaction amounts, and the calculation formula is as follows:

[0123] (19)

[0124] Step S44: The incentive for the power generator to sell carbon quotas depends on the carbon quota interaction volume and carbon quota coefficient of the power generator. The calculation formula is as follows:

[0125] (20)

[0126] in, for Time period generator Incentives to participate in carbon quota interactions.

[0127] Step S5: Use the DDPG deep reinforcement learning method to solve the decision-making model of power generators under the electricity-carbon coupling mechanism and find the optimal strategy solution for power generators. The specific implementation method is as follows:

[0128] Step S51, establish a reinforcement learning environment and agent information. Set the agent as the power generator participating in the electricity-carbon coupling interaction mechanism, the environment as the clearing process of the electricity-carbon coupling interaction mechanism, the exploration process as the interaction process between the agent and the environment, the training process as the process of the agent updating the strategy network and value network, and multiple agents are in the same environment, interacting with the environment independently, and using the feedback rewards of the environment to update their own strategies.

[0129] Step S52: Establish a Markov decision process for the DDPG reinforcement learning algorithm. The Markov process can be represented by a five-tuple described, wherein The state space, which contains all possible states that the agent can be in; is the action space, which contains all possible actions of the agent; is the state transition matrix, which describes the probability of the agent transferring from one state to the next state; is the reward function, which is the reward obtained by the agent after taking an action in the current state; is the discount rate, which is used to measure the impact of current rewards on future expected rewards. and For fixed, set the experience replay pool A quaternion storing the interaction process between the agent and the environment .

[0130] Step S53: Establish the power producer intelligent agent state space:

[0131] (twenty one)

[0132] This includes the application period for power generators to participate in the electricity-carbon coupling interaction mechanism , Load demand during the period , Time-of-day power generators Report the power generation losses of each power generation capacity segment , The remaining power generators in the period report the power generation losses of each power generation capacity segment .

[0133] Step S54: Establishing the action space of the power producer agent:

[0134] (twenty two)

[0135] These include Time-of-day power generators Report the losses of each generation capacity segment.

[0136] Step S55: Establishing a power producer agent reward function

[0137] (twenty three)

[0138] These include Time-of-day power generators Participation in the electricity spot interactive mechanism to sell electricity incentives , participate in the carbon emission interactive mechanism to sell carbon quota incentives , Participation in the electricity spot exchange mechanism power generation losses .

[0139] Step S56: Establish a cumulative discounted return function. The goal of the agent is to maximize the cumulative discounted return in the future. The calculation formula is as follows:

[0140] (twenty four)

[0141] The agent continuously interacts with the environment according to this goal, gradually improving its strategy to increase its reward until its reward reaches the maximum value. The recursive formula is derived based on the definition of cumulative discounted return as follows:

[0142] (25)

[0143] According to this formula, the relationship between the goals of intelligent agents at adjacent moments can be established.

[0144] Step S57: Establishing the DDPG algorithm action value function , the calculation formula is as follows:

[0145] (26)

[0146] Among them, the action value function As shown in (3.2), the agent is in the current state Make the following move After that, in the future, according to its strategy function The expected discounted return from the decision. The action value function is only related to the current state and actions And the policy function The future state and action variables are expected to be eliminated. The factors affecting the current status are as follows: The better, The larger the current action The better, The larger the policy function The better it is, the better the actions the agent will take in the future, and the greater the reward for each action. Therefore, the action value function It can not only measure the quality of the current state and action, but also the quality of the agent's strategy.

[0147] Substitute the recursive formula for cumulative discounted return into the action-value function , the Bellman equation of the reinforcement learning algorithm is as follows:

[0148] (27)

[0149] Step S58: Establish a DDPG algorithm agent strategy network. The goal of the strategy network is to maximize the agent cumulative discounted reward function, and the calculation formula is as follows:

[0150] (28)

[0151] DDPG improves the objective function of the policy network through the policy gradient ascent method , expand the policy gradient according to the chain rule of derivatives, and the calculation formula is as follows:

[0152] (29)

[0153] Update the policy network parameters according to the policy gradient. The calculation formula is as follows:

[0154] (30)

[0155] Step S59: Establish the DDPG algorithm agent value network. Output of action value function at each moment Estimates . Use TD algorithm to update the value network parameters. At this moment, the value network observes the actual , using value networks in The TD target is calculated from the estimate of the moment:

[0156] (31)

[0157] TD targets are based in part on The actual observation at that moment, so it is Closer to the true value of the action value function. The loss function is defined as follows:

[0158] (32)

[0159] Use gradient descent to update the parameters of the value network. The calculation formula is as follows:

[0160] (33)

[0161] Step S510: Establish a DDPG algorithm update mechanism. Use a soft update method to ensure the stability of the algorithm. The calculation formula is as follows:

[0162] (34)

[0163] in, are policy network parameters, is the value network parameter, is the target strategy network parameter, is the target value network parameter, is the soft update coefficient.

[0164] Step S511: Establishing a solution process for a power producer decision optimization method based on the DDPG algorithm, including:

[0165] (1) Define the number of training times , Discount Rate , policy network learning rate , value network learning rate , Value network update frequency ,Soft update frequency ;

[0166] (2) Initialize the policy network parameters of each agent , Value Network Parameters , target strategy network parameters and target value network parameters ;

[0167] (3) The agent obtains the current state of the electric-carbon coupling interaction mechanism ;

[0168] (4) The agent makes decisions based on the current state and outputs actions ;

[0169] (5) The electricity-carbon coupling interaction mechanism is cleared and rewards are fed back to each intelligent agent ; and output the state of the next moment ;

[0170] (6) In the experience replay pool Storing quads in ;

[0171] (7) If the current training step number Divisibility , from the experience replay pool Medium batch sampling quad , calculate the target strategy network output action of each agent and target value network output evaluation , update the value network parameters ;

[0172] (8) If the current training step number Divisibility , update the policy network parameters , soft update target value network parameters and target policy network parameters ;

[0173] (9) Repeat steps (3) to (8) until the number of training times reaches the set value. ;

[0174] (10) Output the optimal strategy of each agent after training.

[0175] This embodiment constructs a series of models and optimization algorithms by comprehensively considering the decision-making problems of power generators under the electricity-carbon coupling interaction mechanism. The DDPG deep reinforcement learning method is used to solve this problem, and the optimal strategy solution for power generators in the electricity spot market and the carbon emission market can be found. This method not only helps power generators improve their economic benefits, but also helps promote the green and low-carbon development of the power system.

Claims

1. A decision-making optimization method for power generators considering the electricity-carbon coupling interaction mechanism, characterized in that: The steps include: S1. Construct a decision-making model that takes into account the participation of power generators in the electricity-carbon coupling interaction, determine the objective function and optimal operation constraints of the power generators' participation in the electricity-carbon coupling interaction mechanism, the objective function of the power generators' participation in the electricity-carbon coupling interaction mechanism takes into account the incentives for power generators to sell electricity in the electricity spot market, the incentives for selling carbon quotas in the carbon emission market, and power generation losses, the optimal operation constraints include upper and lower limits on power generation losses to ensure that the power generators' reported losses are within a reasonable range, and monotonicity constraints on the power generation loss curve to confirm that the loss curve reported by the power generators conforms to the actual operating rules; S2. Construct a power spot mechanism clearing model, determine the objective function and optimal operation constraints of the power spot mechanism clearing model, wherein the objective function of the power spot mechanism clearing model is to minimize the total power generation loss of the system, and the optimal operation constraints include node power balance constraints, upper and lower power constraints of generator sets, climbing constraints of generator sets, upper and lower power constraints of branch currents, node voltage phase angle constraints, and balanced node voltage phase angle constraints; S3. Based on the clearing results of the electricity spot mechanism, free carbon quotas are allocated according to the generating capacity of each power producer’s units, and a carbon quota allocation model is constructed; S4. Based on the clearing results of the electricity spot mechanism and the carbon quota allocation results, combined with the carbon emission intensity of each power company's units, a step-by-step carbon price is used to construct a carbon emission mechanism clearing model; S5. Use the DDPG deep reinforcement learning method to solve the decision-making model of power generators under the electricity-carbon coupling mechanism and seek the optimal strategy.

2. The power generator decision optimization method considering the electricity-carbon coupling interaction mechanism according to claim 1 is characterized in that: In step S1, the objective function of the power generator participating in the electricity-carbon coupling interaction mechanism includes the total incentive of the power generator, which is composed of the power generator's incentive for selling electricity through the electricity spot interaction mechanism, the incentive for selling carbon quotas through the carbon emission interaction mechanism, and the power generation loss through the electricity spot interaction mechanism. The formula is as follows: Where: Provide total incentives for generators; Provide incentives for power generators to participate in the electricity spot exchange mechanism and sell electricity; Incentives for power generators to participate in the carbon emission exchange mechanism and sell carbon quotas; To reduce power generation losses for power generators participating in the electricity spot interaction mechanism.

3. The power generator decision optimization method considering the electricity-carbon coupling interaction mechanism according to claim 1 is characterized in that: In step S2, the objective function of the electricity spot mechanism clearing model is as follows: Where: Reporting for generators Time period generator No. The loss of power generation capacity, for Time period generator No. The output of the power generation capacity of the segment.

4. The power generator decision optimization method considering the electricity-carbon coupling interaction mechanism according to claim 1 is characterized in that: The step S3 constructs a carbon quota allocation model, which specifically includes: S31. Calculate the total amount of carbon quota allowed to be emitted , which is related to the total power generation of the generator set, and the formula is as follows: Where: is the carbon quota benchmark coefficient, for Time period generator No. Output of the generating capacity of the segment; S32. Allocate free carbon quotas to each power generator based on the clearing results of the electricity spot mechanism , the formula is as follows: Where: for Time period generator The free carbon quota allocation coefficient.

5. The power generator decision optimization method considering the electricity-carbon coupling interaction mechanism according to claim 1 is characterized in that: The step S4 constructs a carbon emission mechanism clearing model, specifically including: S41: Calculate the actual carbon emissions of power generation by the power generator using the following formula: In the formula, for Time period generator Actual carbon emissions from electricity generation, For generator sets Carbon emission intensity per unit of electricity generation; S42: Calculate the carbon quota interaction of the power generator, that is, the difference between the actual carbon emissions and the free carbon quota, the formula is as follows: In the formula, yes Time period generator Carbon quotas participating in carbon quota interactions; S43: Set a step-by-step carbon quota coefficient, and set different carbon quota coefficients for different carbon quota interaction amounts. The formula is as follows: S44: Calculate the incentive for power generators to sell carbon quotas using the following formula: In the formula, for Time period generator Incentives to participate in carbon quota interactions.

6. The power generator decision optimization method considering the electricity-carbon coupling interaction mechanism according to claim 1 is characterized in that: The step S5 is solved by using the DDPG deep reinforcement learning method, which specifically includes: S51: Establish a reinforcement learning environment and agent information, taking the power generator as the agent and the clearing process of the electricity-carbon coupling interaction mechanism as the environment; S52: Establish the Markov decision process of DDPG reinforcement learning algorithm to describe the dynamic process of interaction between the agent and the environment; S53: Establish the state space of the power generator agent: The formula includes the reporting period for power generators to participate in the electricity-carbon coupling interaction mechanism. , Load demand during the period , Time-of-day power generators Report the power generation losses of each power generation capacity segment , The remaining power generators in the period report the power generation losses of each power generation capacity segment ; S54: Establish the action space of the power producer agent: In the formula, including Time-of-day power generators Declare the losses of each generation capacity segment; S55: Establish a reward function to motivate the agent to adopt the optimal strategy; S56: Establish a cumulative discounted reward function and an action value function to evaluate the quality of the agent's strategy; S57: Establish the policy network and value network respectively, and update the network parameters by gradient ascent and gradient descent methods; S58: Establish a solution process for the power generator decision optimization method based on the DDPG algorithm, and finally output the optimal strategy for each intelligent agent through multiple iterative training.

7. The power generator decision optimization method considering the electricity-carbon coupling interaction mechanism according to claim 6 is characterized in that: The step S58: establishing a power producer decision optimization method based on the DDPG algorithm, including: (1) Define the number of training times , Discount Rate , policy network learning rate , value network learning rate , Value network update frequency ,Soft update frequency ; (2) Initialize the policy network parameters of each agent , Value Network Parameters , target strategy network parameters and target value network parameters ; (3) The agent obtains the current state of the electric-carbon coupling interaction mechanism ; (4) The agent makes decisions based on the current state and outputs actions ; (5) The electricity-carbon coupling interaction mechanism is cleared and rewards are fed back to each intelligent agent ; and output the state of the next moment ; (6) In the experience replay pool Storing quads in ; (7) If the current training step number Divisibility , from the experience replay pool Medium batch sampling quad , calculate the target strategy network output action of each agent and target value network output evaluation , update the value network parameters ; (8) If the current training step number Divisibility , update the policy network parameters , soft update target value network parameters and target policy network parameters ; (9) Repeat steps (3) to (8) until the number of training times reaches the set value. ; (10) Output the optimal strategy of each agent after training.

Images