Bidding strategy method and device for electric power spot mechanism

Abstract

The invention relates to the technical field of power regulation and control, and discloses a bidding strategy method and device for a power spot mechanism. The method comprises the following steps: constructing a day-ahead unit commitment clearing model and a day-ahead economic dispatching clearing model; constructing a reward function; acquiring initial state information of each power generator; solving an initial bidding strategy of each power generator based on an MADDPG algorithm; based on the initial bidding strategy of each power generator, the day-ahead unit combination clearing model and the day-ahead economic dispatching clearing model, obtaining state information of each power generator at the next moment; obtaining an action reward value of the initial bidding strategy based on a reward function; and iteratively training network parameters of an MADDPG algorithm based on the state information and the action reward value of each power generator at the next moment, and determining an optimal bidding strategy of each power generator. The optimal bidding strategy of the generator is solved under the influence of medium and long term contracts, the power supply adequacy and stability of the power system are improved, and the power shortage and imbalance risk of the power system is reduced.

Description

Technical Field

[0001] The present disclosure relates to the technical field of power regulation and control, and in particular to a bidding strategy method and device for a power spot market mechanism. Background Art

[0002] With the continued growth in energy and electricity demand and the increasing proportion of renewable energy generation connected to the grid, the global energy structure and traditional power systems are accelerating their evolution towards a new power system characterized by "clean power sources, intelligent grids, and electrified users," as well as a corresponding power mechanism system. During this energy transition, the integration of a high proportion of renewable energy and the rigid growth of electricity demand during peak hours have created a temporal mismatch between renewable energy generation and user load-side electricity consumption. The conflicting nature of renewable energy generation, coupled with the uncertainty of load-side electricity consumption, means that the risk of periodic power shortages and power system imbalances persists. This, to a certain extent, necessitates the necessity and regularity of ensuring stable power supply and prices during peak hours.

[0003] The global crisis of energy shortage, sharp price increases, and insufficient supply, coupled with the recent tight power balance and hard gap trend in China due to the peak season, have further intensified the urgency of ensuring power supply based on the reliability and adequacy of the power grid for energy security, while taking into account the interests of all participants and the social responsibility of fair electricity use.

[0004] Medium- and long-term contracts can help power generators and power users sign power contracts in advance, reducing the risks and uncertainties associated with the spot electricity market. Therefore, studying the relationship between medium- and long-term contracts and the spot electricity market is crucial for ensuring sufficient and stable power supply in the power system. Summary of the Invention

[0005] In response to the above situation, the embodiments of the present application provide a bidding strategy method and device for an electricity spot market mechanism, aiming to solve the above problem or at least partially solve the above problem.

[0006] In a first aspect, an embodiment of the present application provides a bidding strategy method for an electricity spot market mechanism, the method comprising: constructing a day-ahead unit portfolio clearing model and a day-ahead economic dispatch clearing model for each power generator;

[0007] The reward function for each generator's bidding strategy is constructed based on the generator's power sales incentives for participating in medium- and long-term contracts, the generator's power sales incentives for participating in the spot market mechanism, and the generator's bid price for participating in the spot market mechanism.

[0008] Obtaining initial status information for each power generator, including the power generator's declared period t for participating in the electricity spot market mechanism, load demand during period t, medium- and long-term contracted electricity volume broken down into period t, and electricity prices declared by the power generator for each generation capacity segment during period t-1;

[0009] Based on the MADDPG algorithm and the initial state information, the action information of each power producer is obtained and the initial bidding strategy of each power producer is determined. The action information includes the electricity price declared by the power producer for each generation capacity segment during period t.

[0010] Based on the initial bidding strategies of each power producer, the day-ahead unit commitment clearing model, and the day-ahead economic dispatch clearing model, the state information of each power producer at the next moment is obtained;

[0011] Obtain action reward values of the initial bidding strategy based on the reward function;

[0012] Based on the state information and action reward value of each power producer at the next moment, the network parameters of the MADDPG algorithm are iteratively trained to adjust the bidding strategy of each power producer until the optimal bidding strategy for each power producer is found.

[0013] In a second aspect, an embodiment of the present application further provides a bidding strategy device for an electricity spot market mechanism, the device comprising:

[0014] A construction module is used to construct a day-ahead unit commitment clearing model and a day-ahead economic dispatch clearing model for each generator. The reward function for each generator's bidding strategy is constructed based on the generator's power sales incentives for participating in medium- and long-term contracts, the generator's power sales incentives for participating in the spot market mechanism, and the generator's bid price in the spot market mechanism.

[0015] An acquisition module is used to obtain the initial status information of each power generator, including the power generator's application period t for participating in the electricity spot market mechanism, the load demand during period t, the medium- and long-term contracted electricity volume decomposed into period t, and the electricity price of each generation capacity segment applied by the power generator during period t-1;

[0016] A processing module is configured to obtain action information of each generator based on the MADDPG algorithm and initial state information, and determine the initial bidding strategy of each generator. The action information includes the electricity price declared by the generator for each generation capacity segment during time period t; obtain the state information of each generator at the next moment based on the initial bidding strategy of each generator, the day-ahead unit commitment clearing model, and the day-ahead economic dispatch clearing model; and obtain the action reward value of the initial bidding strategy based on the reward function;

[0017] The training module is used to iteratively train the network parameters of the MADDPG algorithm based on the state information and action reward value of each power producer at the next moment, and adjust the bidding strategy of each power producer until the optimal bidding strategy for each power producer is found.

[0018] In a third aspect, an embodiment of the present application further provides an electronic device comprising: a processor; and a memory arranged to store computer-executable instructions, wherein the executable instructions, when executed, cause the processor to perform the steps of the first aspect described above.

[0019] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple applications, the electronic device performs the steps of the first aspect above.

[0020] At least one of the above technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: when determining the bidding strategy of each power producer participating in the spot market mechanism, the MADDPG algorithm is used to solve the optimal bidding strategy of each power producer. When finding the optimal bidding strategy for each power producer, the action choices of other power producers are taken into account. This allows for adapting to complex interactive environments and quickly determining the bidding strategy of each power producer agent. Furthermore, when setting the reward function of the MADDPG algorithm, the medium- and long-term contracts of each power producer are taken into account, ensuring the sufficient and stable power supply of the power system. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0022] Figure 1 A schematic diagram showing a flow chart of a bidding strategy method for the electricity spot market mechanism provided in an embodiment of the present application is shown;

[0023] Figure 2 A structural diagram of a bidding strategy device for a power spot market mechanism provided in an embodiment of the present application is shown;

[0024] Figure 3 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0026] It should be noted that the terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that such usage is interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the term "including" and its variations are to be interpreted as open-ended terms meaning "including but not limited to."

[0027] As described in the background art, studying the power generation ratio relationship between medium- and long-term contracts and spot mechanisms is of great significance for ensuring the adequacy and stability of power supply in the power system.

[0028] Equilibrium analysis, based on game theory, studies the interactions and influences among multiple participants in a given mechanism framework as they pursue their own self-interest, ultimately determining the Nash equilibrium state they achieve. The power mechanism equilibrium analysis method can provide a quantitative reference and evaluation for analyzing fireworks under different mechanism designs, making it an important tool for comparing mechanism designs and evaluating their effectiveness.

[0029] While traditional model-based approaches have achieved remarkable success in solving equilibrium solutions for linear models in electricity spot market mechanisms, in the context of new power system construction, existing equilibrium analysis models and solution methods struggle to meet the higher demands of power mechanism design. Traditional equilibrium analysis methods struggle to address the non-convex nature of generator unit costs and operating characteristics under optimality constraints, and the number of complementary slack conditions in the model linearization step increases dramatically with system scale and mechanism complexity. This presents significant challenges for these approaches in addressing equilibrium problems in large-scale systems and complex mechanisms.

[0030] In addition, the model is essentially a complete information game, in which each subject has complete information about other subjects and clearing, which is contrary to the limited information of the actual power mechanism, resulting in the equilibrium solution obtained being unable to reflect the real mechanism.

[0031] Based on this, the present invention proposes a balanced solution method for the electricity spot market mechanism.

[0032] The present application is described in detail below through specific embodiments.

[0033] Figure 1 The flow chart of the method for balancing the electricity spot market mechanism provided by the embodiment of the present application is shown. Figure 1 It can be seen that this application at least includes steps S101 to S103:

[0034] Step S101: Constructing a day-ahead unit combination clearing model and a day-ahead economic dispatch clearing model for each power producer.

[0035] Step S102: Constructing a reward function for each generator's bidding strategy based on the generator's power sales incentives for participating in medium- and long-term contracts, the generator's power sales incentives for participating in the spot market mechanism, and the generator's bids for participating in the spot market mechanism.

[0036] Step S103: Acquire the initial status information of each power generator.

[0037] The status information includes the power generation company's application period t for participating in the electricity spot market mechanism, the load demand in period t, the medium- and long-term contracted electricity volume decomposed into period t, and the electricity price of each generation capacity segment applied by the power generation company in period t-1;

[0038] Step S104: Based on the MADDPG algorithm and the initial state information, the action information of each power producer is obtained and the initial bidding strategy of each power producer is determined.

[0039] The action information includes the electricity prices of each generation capacity segment declared by the power generator during period t;

[0040] Step S105: Based on the initial bidding strategy of each power producer, the day-ahead unit commitment clearing model, and the day-ahead economic dispatch clearing model, the state information of each power producer at the next moment is obtained.

[0041] Step S106: Obtain action reward values of the initial bidding strategy based on the reward function.

[0042] Step S107: Iteratively train the network parameters of the MADDPG algorithm based on the state information and action reward value of each power generator at the next moment, and adjust the bidding strategy of each power generator until the optimal bidding strategy of each power generator is found.

[0043] Furthermore, the equilibrium solution of the electricity spot market mechanism is determined through the optimal bidding strategy of power producers.

[0044] from Figure 1 As can be seen from the method shown, this application uses the MADDPG algorithm to solve the optimal bidding strategy for each generator when determining its bidding strategy for participating in the spot market mechanism. This algorithm considers the actions of other generators when finding the optimal bidding strategy for each generator, adapting to complex interactive environments and quickly determining the bidding strategy for each generator agent. Furthermore, when setting the reward function for the MADDPG algorithm, the medium- and long-term contracts of each generator are taken into account, ensuring sufficient power supply and stability for the power system.

[0045] In some embodiments of the present application, in the above step S101, the day-ahead unit combination clearing model includes a day-ahead unit combination clearing function and constraints of the day-ahead unit combination clearing function. Specifically, the objective function of the day-ahead unit combination model is determined based on the power system operation boundary conditions and the unit information reported by the power generator. The power system operation boundary conditions include the total number of generators, the total number of branches, the total number of sections, and the total number of time periods. The unit information reported by the power generator includes the operating loss of unit i in time period t, the starting loss of unit i in time period t, the no-load loss of unit i in time period t, the network power flow constraint penalty factor, the forward power flow slack variable of the branch, the reverse power flow slack variable of the branch, the forward power flow slack variable of the node, and the reverse power flow slack variable of the node.

[0046] In some embodiments, the objective function of the day-ahead unit commitment model is as follows:

[0047]

[0048] Among them, N is the total number of generator sets, N L is the total number of branches, N S is the total number of sections, T is the total number of time periods, P i,t represents the generating power of unit i during period t, X i,t (P i,t ) is the operating loss of unit i during period t, is the starting loss of unit i during period t, is the no-load loss of unit i during period t, M is the penalty factor for network power flow constraint, is the forward power flow slack variable of branch l, is the reverse power flow slack variable of branch l, is the forward power flow slack variable of node s, is the reverse power flow slack variable of node s.

[0049] In some embodiments, the constraints of the clearing model of the day-ahead unit combination include system power balance constraints, system spare capacity constraints, system rotating reserve constraints, unit output upper and lower limit constraints, unit ramp constraints, unit minimum continuous start and stop time constraints, unit maximum start and stop number constraints, branch flow constraints, and section flow constraints.

[0050] Specifically, the formulas for each constraint condition are as follows:

[0051] (1) System power balance constraints:

[0052]

[0053] Among them, P i,t represents the generating power of unit i during period t, N T is the number of tie lines, Tj,t is the planned power of tie line j in period t, D t is the system load during period t.

[0054] (2) System reserve capacity constraints, including positive reserve constraints and negative reserve constraints:

[0055]

[0056] Among them, α i,t is the start and stop status of generator set i during period t, is the upper limit of the power generation capacity of generator set i during period t, is the lower limit of the power generation of generator set i during period t, is the system standby requirement for period t, are the system negative backup requirements for period t respectively.

[0057] (3) System spinning reserve constraints: the sum of the units' upward and downward adjustment capabilities in each period must meet the actual operation's upward and downward spinning reserve requirements:

[0058]

[0059] in, is the maximum ramp rate of generator set i, is the maximum ramp rate of generator set i, Increase the spinning reserve requirement for the system during period t, Adjust the spinning reserve requirement downward for the system during period t.

[0060] (4) Upper and lower limits of unit output:

[0061]

[0062] The output of the unit should be within the range of its maximum technical output and minimum technical processing.

[0063] (5) Unit climbing constraints:

[0064]

[0065] The ramp constraint stipulates that the difference in the unit's generated power in adjacent time periods should not exceed the maximum upward ramp rate or the maximum downward ramp rate.

[0066] (6) The minimum continuous start and stop time constraint of the unit. Due to the physical properties of the thermal power unit, it is required to meet the minimum continuous start and stop time during actual operation, as follows:

[0067]

[0068] Among them, T U and TD They are the minimum continuous operating time and the minimum continuous shutdown time of the unit, and are the continuous start-up time and continuous shutdown time of generator set i in period t respectively.

[0069] (7) Constraints on the maximum number of starts and stops of the unit:

[0070]

[0071] in, and are the maximum startup switching times and the maximum shutdown switching times of the unit, η i,t and γ i,t They are the start-up switching state and shutdown switching state of generator set i in period t respectively.

[0072] (8) Branch flow constraints:

[0073]

[0074] Where K is the number of system nodes, and are the transmission capacity limit of branch l, G l-i is the generator output power transfer distribution factor of the node where unit i is located to branch l, G l-j is the generator output power transfer distribution factor of the node where the tie line j is located to the branch l, G l-k is the generator output power transfer distribution factor of node k to branch l, d k,t is the bus load value of node k in period t.

[0075] (9) Sectional tidal flow constraints:

[0076]

[0077] Where K is the number of system nodes, is the maximum transmission capacity of section s, is the minimum transmission capacity of section s, G s-i is the generator output power transfer distribution factor of the node where unit i is located on section s, G s-j is the generator output power transfer distribution factor of the node where the tie line j is located on section s, G s-k is the generator output power transfer distribution factor for node k to section s.

[0078] In some embodiments of the present application, a day-ahead unit combination clearing result is determined based on a day-ahead unit combination clearing model, and a day-ahead economic dispatch clearing model is constructed based on the day-ahead unit combination clearing result.

[0079] In some embodiments, the objective function of the day-ahead economic dispatch clearing model is as follows:

[0080]

[0081] Among them, N is the total number of generator sets, N L is the total number of branches, N S is the total number of sections, T is the total number of time periods, P i,t represents the power generation of unit i during period t, C i,t (P i,t ) is the operating loss of unit i during period t, M is the penalty factor for network power flow constraint, is the forward power flow slack variable of branch l, is the reverse power flow slack variable of branch l, is the forward power flow slack variable of node s, is the reverse power flow slack variable of node s.

[0082] In some embodiments, the constraints of the day-ahead economic dispatch clearing model include system power balance constraints, system spinning reserve constraints, unit output upper and lower limit constraints, unit ramping constraints, branch flow constraints, and section flow constraints.

[0083] Specifically, the formulas for each constraint condition are as follows:

[0084] (1) System power balance constraints:

[0085]

[0086] Among them, N T is the number of tie lines, T j,t is the planned power of tie line j in period t, D t is the system load during period t.

[0087] (2) System spinning reserve constraint: The sum of the upward and downward regulating capabilities of the units in each period must meet the upward and downward regulating spinning reserve requirements of actual operation, as shown in the following formula:

[0088]

[0089] in, and are the maximum ramp-up rate and the maximum ramp-down rate of generator set i, respectively. and They are the upward and downward adjustment of the system spinning reserve requirements during period t respectively.

[0090] (3) The upper and lower limits of the unit output are constrained. The unit output should be within the range of its maximum technical output and minimum technical processing capacity:

[0091]

[0092] (4) Unit ramp constraint: The ramp constraint stipulates that the difference in the unit's power generation in adjacent periods should not exceed the maximum ramp rate or the maximum ramp rate:

[0093]

[0094] (5) Branch flow constraints:

[0095]

[0096] Where K is the number of system nodes, and are the transmission capacity limit of branch l, G l-i is the generator output power transfer distribution factor of the node where unit i is located to branch l, G l-j is the generator output power transfer distribution factor of the node where the tie line j is located to the branch l, G l-k is the generator output power transfer distribution factor of node k to branch l, D k,t is the bus load value of node k in period t.

[0097] (6) Sectional tidal flow constraints:

[0098]

[0099] Where K is the number of system nodes, and are the transmission capacity limit of section s, G s-i is the generator output power transfer distribution factor of the node where unit i is located on section s, G s-j is the generator output power transfer distribution factor of the node where the tie line j is located on section s, G s-k is the generator output power transfer distribution factor for node k to section s.

[0100] (7) Node electricity price calculation:

[0101]

[0102] Among them, λ t is the Lagrange multiplier for the system power balance constraint, and are the Lagrange multipliers for the maximum positive and negative power flow constraints of branch l, and are the Lagrange multipliers for the maximum positive and negative tidal flow constraints of section s, respectively.

[0103] In some embodiments of the present application, in the above method S102, a reward function for each power generator's bidding strategy is constructed based on the power generator's participation in medium- and long-term power sales incentives, the power generator's participation in the spot mechanism's power sales incentives, and the power generator's quotation for participating in the spot mechanism.

[0104] In some embodiments, a bidding model for each generator participating in the spot market mechanism is pre-established. In one embodiment, the bidding model for each generator participating in the spot market mechanism is established based on the generator's mid- to long-term electricity sales incentives, the generator's electricity sales incentives for participating in the spot market mechanism, and the generator's bid price for participating in the spot market mechanism.

[0105] Specifically, the bidding goal is to maximize the total incentives of each power producer, so the bidding model for each power producer participating in the spot market mechanism is as follows:

[0106] max R=R C +R S -C G

[0107] Among them, R is the total incentive for power generators, R G To provide incentives for power generators to participate in medium- and long-term contracts, R S To provide incentives for power generators to participate in the spot market mechanism, C G The electricity price declared by power generators participating in the spot mechanism.

[0108] Furthermore, the power sales incentives for power generators participating in medium- and long-term contracts are determined based on the electricity volume and electricity price signed by the power generators in the medium- and long-term contracts; the power sales incentives for power generators participating in the spot mechanism are determined based on the electricity price of the node where the power generator is located in each time period, the power generation power of the power generator in each time period in the spot mechanism, and the contracted electricity volume decomposed into each time period, wherein the contracted electricity volume decomposed into each time period is determined based on the system load in each time period and the electricity volume signed by the power generator in the medium- and long-term contracts; the quotation of power generators participating in the spot mechanism is determined based on the quotation curve coefficient of the power generation unit and the power generation power of the power generator in each time period.

[0109] Specifically, the power sales incentives for generators participating in medium- and long-term contracts are determined based on the following formula:

[0110] R C =λ i C P i C

[0111] in, is the amount of electricity signed by generator i in the medium and long-term contract, is the electricity price signed by generator i in the medium and long-term contract.

[0112] The power sales incentive for generators participating in the spot market mechanism is determined based on the following formula:

[0113]

[0114] Among them, λ k,t is the electricity price at node k where generator i is located during period t, P i,t is the power generation of generator i in period t in the spot mechanism, is the contract electricity amount decomposed into period t.

[0115] The contract electricity volume broken down into period t is determined based on the following formula:

[0116]

[0117] Among them, D t is the system load during period t, The power quantity contracted by generator i in a medium- to long-term contract. The system load for each time period is determined based on the short-term load forecast curve. This curve forecasts the power load for the next one to seven days. It can be used to determine the system load for each time period. Specifically, the short-term load forecast curve is derived using various forecasting methods, including time series analysis, neural network prediction, and fuzzy logic.

[0118] In some embodiments, the electricity price reported by the generator set must meet the upper and lower limit constraints:

[0119]

[0120] Among them, α i,b,t is the electricity price declared by generator i for the nth generation capacity in period t, is the maximum declared electricity price for the nth generation capacity of generator i during period t, The minimum declared electricity price for the bth section of generating capacity of generator i in period t.

[0121] At the same time, the electricity price curve must meet the monotonicity constraint:

[0122] α i,b,t ≥α i,b-1,t

[0123] The electricity price curve declared by the power generator should be in a monotonically increasing form, and the electricity price declared for the bth section of generating capacity should be greater than or equal to the electricity price declared for the b-1th section of generating capacity.

[0124] Furthermore, a reward function for each power generator is constructed based on the bidding model of each power generator participating in the spot market mechanism.

[0125] In some embodiments, the reward function is as follows:

[0126]

[0127] in, represents the power sales incentives of generator i participating in medium- and long-term contracts during period t, To provide incentives for electricity sales in the electricity spot market mechanism, Indicates the electricity price declared for participating in the spot mechanism, Signing electricity prices for power generator i in medium and long-term contracts, is the contract electricity quantity decomposed into period t, λ k,t is the electricity price at node k where generator i is located during period t, P i,t is the power generation of generator i in period t in the spot mechanism, a i 、b i 、c i is the declared electricity price curve coefficient of generator i.

[0128] In the embodiment of the present application, when formulating the reward function, the medium and long-term contracts are taken into consideration.

[0129] In some embodiments of this application, each agent acts as a generator participating in the electricity spot market mechanism, and the agent's environment acts as the clearing process for the electricity spot market mechanism. The exploration process is an interactive process between the agent and the environment. Each agent independently interacts with the environment and obtains reward information, without any information transfer between agents. A policy network and a value network are set for each agent, and the training process involves updating the policy network and value network based on the status collected during the exploration process.

[0130] Furthermore, the MADDPG Markov decision process is established. The Markov process can be described by a five-tuple (S, A, P, R, γ). Among them, S is the state space, including the set of states in which the agent is located; A is the state space, including the set of actions that the agent can take; P is the state transition matrix, which describes the probability of the agent switching between states; R is the reward function, which represents the reward obtained by the agent after taking an action in a specific state; γ is the discount rate, which is used to measure the impact of the current reward on the expected reward in the future. Set up the experience replay pool The four-tuple (s) that stores the interaction process between the agent and the environment t ,a t ,r t ,s t+1 ).

[0131] (1) Establish the state space of each agent:

[0132]

[0133] Among them, t represents the declaration period for power generators to participate in the electricity spot market mechanism, represents the load demand during period t, represents the contract electricity volume decomposed into period t, αi,b,t-1 It indicates the electricity price of each capacity segment declared by the power generator during the t-1 period.

[0134] (2) Establish the action space of each agent:

[0135] a t =(α i,1,t ,α i,2,t ,...,α i,B,t )

[0136] Among them, α i,B,t It represents the price of each generation capacity segment B declared by power producer i during period t.

[0137] (3) Establish reward functions for each agent:

[0138]

[0139] in, represents the power sales incentives of generator i participating in medium- and long-term contracts during period t, To provide incentives for electricity sales in the electricity spot market mechanism, Indicates the electricity price declared for participating in the spot mechanism, Signing electricity prices for power generator i in medium and long-term contracts, is the contract electricity quantity decomposed into period t, λ k,t is the electricity price at node k where generator i is located during period t, P i,t is the power generation of generator i in period t in the spot mechanism, a i 、b i 、c i is the coefficient of the power generation company's declared electricity price curve, and the declared electricity price curve is a quadratic function.

[0140] (4) Establishing a cumulative discount reward function

[0141] R t =r t +γr t+1 +γ 2 r t+2 +...+γ T-t r T

[0142] The training and decision-making goal of each agent is to maximize the cumulative discounted return taking into account future rewards. The agent continuously interacts with the environment according to the cumulative discounted return function, gradually improving its strategy to increase its reward until the function value reaches the maximum.

[0143] (5) Establish the MADDPG agent value network q(s,a;ω i ), train the value network to score the agent's actions q(s,a;ω i) is closer to the true action-value function Extract the quadruple group (s) from the experience replay pool t ,a t ,r t ,s t+1 ), s t+1 Input the policy network of each agent and output the action set of all agents in the next state The time difference target is calculated as follows:

[0144]

[0145] in, represents the time difference target value of generator i during period t, represents the action reward value of generator i during period t, γ represents the discount factor, q(s t+1 ,a t+1 ;ω i ) indicates that the t+1 and a t+1 The target value output by the value network when it is input.

[0146] The time difference target and the target value q(s,a;ω i ) for comparison, and the time difference error is calculated as follows:

[0147]

[0148] Among them, q(s t ,a t ;ω i ) table indicates the t and a t The target value output by the value network when it is input.

[0149] According to the time difference error, the network parameters ω of the value network are updated using gradient descent i ,as follows:

[0150]

[0151] (6) Establish the agent strategy network μ(o i θ i ). Train the policy network to improve the score q(s,a;ω) of the agent’s value network. i ), the formula is as follows:

[0152] J i (θ 1 ,...,θ m )=E S [q(S,[μ(O 1 θ1 ),...,μ(O m θ m )];ω i )]

[0153] Make a Monte Carlo approximation of the target of the policy network and calculate the gradient. Then, expand the policy gradient according to the chain rule of derivatives. The calculation formula is as follows:

[0154]

[0155] in, Represents theta i Find the gradient, μ(o m θ m ) indicates that the value network is in state o m Output action when .

[0156] Update the parameters θ according to the gradient ascent i , the calculation formula is as follows:

[0157]

[0158] Step S59: Use the soft update method to update the network parameters of the target network in the strategy network and the target network in the value network at each preset time to ensure the stability of the training process. The calculation formula is as follows:

[0159]

[0160] Among them, θ is the policy network parameter, ω is the value network parameter, θ′ is the network parameter of the target network in the policy network, ω′ is the target value network parameter, and τ∈[0,1] is the soft update coefficient.

[0161] Each generator updates its own network using the same method. The inputs are different for each generator, but the rest of the update process is the same.

[0162] In an embodiment of the present application, each power generator specifies action strategy information based on status information, and all power generators submit the action strategy information to the environmental market for market clearing and settlement, and feedback the rewards and updated status information to each intelligent agent. The power generator adjusts its own strategy based on the latest status information and the obtained reward value. Through multiple iterative training, the algorithm can simulate the behavior of power generators and eventually find the optimal bidding strategy for different power generators, that is, maximize the total incentive of each power generator.

[0163] In some embodiments of the present application, a bidding strategy device for an electricity spot market mechanism is provided, and the bidding strategy device for the electricity spot market mechanism corresponds one-to-one to the bidding strategy method for the electricity spot market mechanism in the above embodiment. Figure 2As shown, the bidding strategy device of the electricity spot mechanism includes a construction module 101 , an acquisition module 102 , a processing module 103 and a training module 104 .

[0164] Construction module 101 is used to construct a day-ahead unit commitment clearing model and a day-ahead economic dispatch clearing model for each generator; construct a reward function for each generator's bidding strategy based on the generator's power sales incentives for participating in medium- and long-term contracts, the generator's power sales incentives for participating in the spot market mechanism, and the generator's bid price for participating in the spot market mechanism;

[0165] Acquisition module 102 is used to obtain initial status information of each power generator, including the power generator's application period t for participating in the electricity spot market mechanism, the load demand during period t, the medium- and long-term contracted electricity volume broken down into period t, and the electricity price applied by the power generator for each generation capacity segment during period t-1;

[0166] Processing module 103 is configured to obtain action information of each power producer based on the MADDPG algorithm and initial state information, and determine an initial bidding strategy for each power producer. The action information includes the electricity price declared by the power producer for each generation capacity segment during time period t; obtain the state information of each power producer at the next moment based on the initial bidding strategy of each power producer, the day-ahead unit commitment clearing model, and the day-ahead economic dispatch clearing model; and obtain an action reward value for the initial bidding strategy based on a reward function.

[0167] The training module 104 is used to iteratively train the network parameters of the MADDPG algorithm based on the state information and action reward value of each power producer at the next moment, and adjust the bidding strategy of each power producer until the optimal bidding strategy of each power producer is found.

[0168] In some embodiments of the present application, in the above-mentioned device, the power sales incentive for the power generator participating in the medium- and long-term contracts is determined based on the power quantity and electricity price signed by the power generator in the medium- and long-term contracts; the power sales incentive for the power generator participating in the spot mechanism is determined based on the electricity price of the node where the power generator is located in each time period, the power generation power of the power generator in each time period in the spot mechanism, and the contract power quantity decomposed into each time period, wherein the contract power quantity decomposed into each time period is determined based on the system load of each time period and the power quantity signed by the power generator in the medium- and long-term contracts; the quotation of the power generator participating in the spot mechanism is determined based on the quotation curve coefficient of the power generation group and the power generation power of the power generator in each time period.

[0169] In some embodiments of the present application, in the above device, the reward function is as follows:

[0170]

[0171] in, represents the power sales incentives of generator i participating in medium- and long-term contracts during period t, To provide incentives for electricity sales in the electricity spot market mechanism, Indicates the electricity price declared for participating in the spot mechanism, Signing electricity prices for power generator i in medium and long-term contracts, is the contract electricity quantity decomposed into period t, λ k,t is the electricity price at node j where generator i is located during period t, P i,t is the power generation of generator i in period t in the spot mechanism, a i 、b i 、c i is the declared electricity price curve coefficient of generator group g.

[0172] In some embodiments of the present application, in the above-mentioned device, the processing module 103 is specifically used to determine the day-ahead unit combination clearing result based on the initial bidding strategy and the day-ahead unit combination clearing model; based on the initial bidding strategy, the day-ahead unit combination clearing result and the day-ahead economic dispatch clearing model, obtain the status information of each power generator at the next moment.

[0173] In some embodiments of the present application, in the above-mentioned device, the objective function of the day-ahead unit commitment clearing model is:

[0174]

[0175] Among them, N is the total number of generator sets, N L is the total number of branches, N S is the total number of sections, T is the total number of time periods, P i,t represents the power generation of unit i during period t, C i,t (P i,t ), are the operating loss, starting loss and no-load loss of unit i in period t, M is the network power flow constraint penalty factor, are the forward and reverse power flow slack variables of branch l, are the forward and reverse power flow slack variables of node s respectively;

[0176] The constraints of the day-ahead unit combination clearing model include system power balance constraints, system spare capacity constraints, system spinning reserve constraints, unit output upper and lower limit constraints, unit ramping constraints, unit minimum continuous start and stop time constraints, unit maximum start and stop number constraints, branch power flow constraints, and section power flow constraints.

[0177] In some embodiments of the present application, in the above-mentioned device, the objective function of the day-ahead economic dispatch clearing model is:

[0178]

[0179] Among them, N is the total number of generator sets, N Lis the total number of branches, N S is the total number of sections, T is the total number of time periods, P i,t represents the power generation of unit i during period t, X i,t (P i,t ) is the operating loss of unit i during period t, M is the penalty factor for network power flow constraint, are the forward and reverse power flow slack variables of branch l, are the forward and reverse power flow slack variables of node s respectively; the constraints of the day-ahead economic dispatch clearing model include system power balance constraints, system spinning reserve constraints, unit output upper and lower limit constraints, branch power flow constraints and section power flow constraints.

[0180] In some embodiments of the present application, in the above-mentioned device, each power producer agent in the MADDPG algorithm corresponds to a strategy network and a value network;

[0181] The training module 104 is used to input the status information of each power generator at the next moment and the action information of each power generator at the next moment into the value network to obtain the target value output by the value network at the next moment; calculate the time difference target based on the action reward value and the target value output by the value network; compare the time difference target with the target value output by the value network at time t to obtain the time difference error; update the network parameters of the value network based on the time difference error; use Monte Carlo and the chain rule of derivatives to solve the policy gradient of the policy network; update the network parameters of the policy network based on the policy gradient; update the network parameters of the target network in the policy network and the target network in the value network through the soft update algorithm at preset time intervals.

[0182] It should be noted that any of the above-mentioned bidding strategy devices for the electricity spot market mechanism can implement the above-mentioned bidding strategy method for the electricity spot market mechanism in a one-to-one correspondence, which will not be repeated here.

[0183] Figure 3 FIG. 1 shows a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 3 As shown, at the hardware level, the electronic device includes a processor and, optionally, an internal bus, a network interface, and memory. The memory may include internal memory, such as high-speed random-access memory (RAM), and may also include non-volatile memory, such as at least one disk storage device. Of course, the electronic device may also include other hardware required for its services.

[0184] The processor, network interface, and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 3 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0185] The memory is used to store programs. Specifically, the program may include program code, which includes computer operating instructions. The memory may include internal memory and non-volatile memory, and provides instructions and data to the processor.

[0186] The processor reads the corresponding computer program from the non-volatile memory into the internal memory and then runs it, forming a bidding strategy device for the electricity spot market mechanism at the logical level. The processor executes the program stored in the memory and is specifically used to perform the aforementioned method.

[0187] The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor or by software instructions. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0188] The electronic device can execute the bidding strategy method of the power spot mechanism provided by multiple embodiments of the present application, and realize the bidding strategy device of the power spot mechanism in Figure 2 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0189] An embodiment of the present application also proposes a computer-readable storage medium, which stores one or more programs, and the one or more programs include instructions. When the instructions are executed by an electronic device including multiple application programs, the electronic device can execute the bidding strategy method of the electricity spot mechanism provided by multiple embodiments of the present application.

[0190] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0191] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0192] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0193] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0194] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0195] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0196] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0197] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0198] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0199] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A bidding strategy method for the electricity spot market mechanism, characterized in that: The method comprises: Construct the day-ahead unit portfolio clearing model and the day-ahead economic dispatch clearing model for each power producer; The reward function for each generator's bidding strategy is constructed based on the generator's power sales incentives for participating in medium- and long-term contracts, the generator's power sales incentives for participating in the spot market mechanism, and the generator's bid price for participating in the spot market mechanism. Obtaining initial status information for each power generator, including the power generator's declared period t for participating in the electricity spot market mechanism, load demand during period t, medium- and long-term contracted electricity volume broken down into period t, and electricity prices declared by the power generator for each generation capacity segment during period t-1; Based on the MADDPG algorithm and the initial state information, the action information of each power producer is obtained and the initial bidding strategy of each power producer is determined. The action information includes the electricity price declared by the power producer for each generation capacity segment during period t. Based on the initial bidding strategies of each power producer, the day-ahead unit commitment clearing model, and the day-ahead economic dispatch clearing model, the state information of each power producer at the next moment is obtained; Obtain action reward values of the initial bidding strategy based on the reward function; Based on the state information and action reward value of each power producer at the next moment, the network parameters of the MADDPG algorithm are iteratively trained to adjust the bidding strategy of each power producer until the optimal bidding strategy for each power producer is found.

2. The method according to claim 1, characterized in that The power sales incentives for power generators participating in medium- and long-term contracts are determined based on the power volume and electricity prices signed by the power generators in the medium- and long-term contracts; The power sales incentive for the generator participating in the spot market mechanism is determined based on the electricity price at the node where the generator is located in each time period, the power generation of the generator in each time period in the spot market mechanism, and the contracted electricity volume decomposed into each time period, wherein the contracted electricity volume decomposed into each time period is determined based on the system load in each time period and the electricity volume signed by the generator in the medium- and long-term contracts; The bids of the power generators participating in the spot market mechanism are determined based on the bid curve coefficients of the power generators and the power generation of the power generators in each period.

3. The method according to claim 1 or 2, characterized in that The reward function is as follows: in, represents the power sales incentives of generator i participating in medium- and long-term contracts during period t, To provide incentives for electricity sales in the electricity spot market mechanism, Indicates the electricity price declared for participating in the spot mechanism, Signing electricity prices for power generator i in medium and long-term contracts, is the contract electricity quantity decomposed into period t, λ k,t is the electricity price at node k where generator i is located during period t, P i,t is the power generation of generator i in period t in the spot mechanism, a i 、b i 、c i is the declared electricity price curve coefficient of generator group g.

4. The method according to claim 1, wherein The state information of each power producer at the next moment is obtained based on the initial bidding strategy of each power producer, the day-ahead unit commitment clearing model, and the day-ahead economic dispatch clearing model, including: Determine the day-ahead unit portfolio clearing result based on the initial bidding strategy and the day-ahead unit portfolio clearing model; Based on the initial bidding strategy, the day-ahead unit commitment clearing results, and the day-ahead economic dispatch clearing model, the status information of each generator at the next moment is obtained.

5. The method according to claim 1 or 4, characterized in that The objective function of the day-ahead unit commitment clearing model is: Among them, N is the total number of generator sets, N L is the total number of branches, N S is the total number of sections, T is the total number of time periods, P i,t represents the power generation of unit i during period t, C i,t (P i,t ), are the operating loss, starting loss and no-load loss of unit i in period t, M is the network power flow constraint penalty factor, are the forward and reverse power flow slack variables of branch l, are the forward and reverse power flow slack variables of node s respectively; The constraints of the day-ahead unit combination clearing model include system power balance constraints, system spare capacity constraints, system spinning reserve constraints, unit output upper and lower limit constraints, unit ramping constraints, unit minimum continuous start and stop time constraints, unit maximum start and stop number constraints, branch power flow constraints, and section power flow constraints.

6. The method according to claim 1 or 4, characterized in that The objective function of the day-ahead economic dispatch clearing model is: Among them, N is the total number of generator sets, N L is the total number of branches, N S is the total number of sections, T is the total number of time periods, P i,t represents the power generation of unit i during period t, C i,t (P i,t ) is the operating loss of unit i during period t, M is the penalty factor for network power flow constraint, are the forward and reverse power flow slack variables of branch l, are the forward and reverse power flow slack variables of node s respectively; The constraints of the day-ahead economic dispatch clearing model include system power balance constraints, system spinning reserve constraints, unit output upper and lower limit constraints, branch power flow constraints, and section power flow constraints.

7. The method according to claim 1, characterized in that In the MADDPG algorithm, each power producer agent corresponds to a strategy network A and a value network C; The network parameters of the MADDPG algorithm are iteratively trained based on the state information and action reward values of each generator at the next moment, including: Input the state information and action information of each power generator at the next moment into the value network to obtain the target value output by the value network at the next moment; Calculate the temporal difference target based on the action reward value and the target value output by the value network; Compare the time difference target with the target value output by the value network at time t to obtain the time difference error; Update the network parameters of the value network based on the time difference error; Use Monte Carlo and the chain rule of derivatives to solve the policy gradient of the policy network; Update the network parameters of the policy network based on policy gradient; The network parameters of the target network in the strategy network and the target network in the value network are updated through the soft update algorithm at preset time intervals.

8. A bidding strategy device for an electricity spot market mechanism, characterized in that: The device comprises: A construction module is used to construct a day-ahead unit commitment clearing model and a day-ahead economic dispatch clearing model for each generator. The reward function for each generator's bidding strategy is constructed based on the generator's power sales incentives for participating in medium- and long-term contracts, the generator's power sales incentives for participating in the spot market mechanism, and the generator's bid price in the spot market mechanism. An acquisition module is used to obtain the initial status information of each power generator, including the power generator's application period t for participating in the electricity spot market mechanism, the load demand during period t, the medium- and long-term contracted electricity volume decomposed into period t, and the electricity price of each generation capacity segment applied by the power generator during period t-1; A processing module is configured to obtain action information of each generator based on the MADDPG algorithm and initial state information, and determine the initial bidding strategy of each generator. The action information includes the electricity price declared by the generator for each generation capacity segment during time period t; obtain the state information of each generator at the next moment based on the initial bidding strategy of each generator, the day-ahead unit commitment clearing model, and the day-ahead economic dispatch clearing model; and obtain the action reward value of the initial bidding strategy based on the reward function; The training module is used to iteratively train the network parameters of the MADDPG algorithm based on the state information and action reward value of each power producer at the next moment, and adjust the bidding strategy of each power producer until the optimal bidding strategy for each power producer is found.

9. An electronic device comprising: processor; as well as A memory arranged to store computer-executable instructions, wherein when the executable instructions are executed, the processor is caused to perform the steps of the bidding strategy method for the electricity spot market mechanism as described in any one of claims 1-7.

10. A computer-readable storage medium storing one or more programs, characterized in that: When the one or more programs are executed by an electronic device including a plurality of application programs, the electronic device executes the steps of the bidding strategy method for the electricity spot market mechanism as described in any one of claims 1 to 7.

Images