Electricity-carbon-green certificate market balanced scheduling method based on multi-agent evolutionary game

Abstract

The invention relates to an electricity-carbon-green certificate market balanced scheduling method based on a multi-agent evolutionary game, and the method comprises the following steps: 1, collecting data, and constructing an electricity-carbon-green certificate three-market double-layer coupling model; 2, carrying out multi-agent modeling, and modeling a power generation enterprise, a power user, a renewable energy enterprise and a supervision mechanism into a multi-agent model; 3, performing evolution modeling on the intelligent agent strategy by adopting a replicated dynamic equation, and constructing an evolutionary game model; 4, a multi-agent near-end strategy optimization algorithm MAPPO is introduced, a multi-agent reinforcement learning model is constructed, and all agents are trained under a'centralized training and distributed execution 'framework; and step 5, after training convergence, outputting an optimal strategy set of each market subject, and obtaining an equilibrium point of the power-carbon-green evidence coupling market. According to the method, the limitation that a traditional method depends on complete rationality and analytical solution is broken through, rapid convergence and stable equilibrium of the complex coupling market are achieved, and the method has remarkable engineering value and policy guidance significance.

Description

Technical Field

[0001] This invention belongs to the field of power market optimization and intelligent decision-making technology, and relates to a power-carbon-green certificate market equilibrium scheduling method, especially a power-carbon-green certificate coupled market equilibrium scheduling method based on multi-agent evolutionary game. Background Technology

[0002] Currently, the coupled market equilibrium dispatch of electricity, carbon, and green certificates is one of the key technologies supporting the low-carbon operation and market-oriented construction of new power systems. Existing research mainly revolves around traditional game theory and optimization methods, such as using Nash equilibrium and Stankelberg game models to model the behavior of market participants and constructing market clearing models based on optimization theory. These methods have certain theoretical value under idealized assumptions, but they have significant shortcomings in real-world complex coupled market environments. Traditional methods typically rely on the assumption of perfect rationality, where all participants have complete information and pursue instantaneously optimal strategies. This differs significantly from the decision-making behavior of actual market participants under conditions of information asymmetry and bounded rationality. With the deep coupling of the electricity market, carbon market, and green certificate market, the model dimensions increase dramatically. Traditional analytical methods struggle to efficiently solve for the global equilibrium point, often getting trapped in local optima, and have low computational efficiency, making it difficult to support large-scale real-time market analysis.

[0003] In recent years, the combination of multi-agent reinforcement learning and evolutionary game theory has provided new insights into market equilibrium analysis. By modeling market participants as agents and introducing a strategy evolution mechanism, the dynamic interaction process of participants can be simulated more realistically within a bounded rationality framework. However, existing intelligent methods still have several key shortcomings when applied to the coupled scenarios of the electricity-carbon-green certificate market: Firstly, multi-agent modeling lacks a unified standard, with significant differences in the definitions of state space, action space, and payoff function across different studies, leading to insufficient model reproducibility and generalization ability. Secondly, reinforcement learning algorithms still need improvement in hyperparameter sensitivity and training stability, especially in high-dimensional state and action spaces where policy oscillations or slow convergence are prone to occur, affecting the efficiency and reliability of equilibrium scheduling. Furthermore, existing research mainly focuses on the coupling of the electricity-carbon dual market and has not systematically considered the profound impact of the green certificate trading mechanism on market participant strategies and market equilibrium, making it difficult to comprehensively reflect the market operation characteristics driven by multiple policies.

[0004] In summary, the existing technology still has the following shortcomings: 1. The assumption of rationality is unrealistic: Traditional methods rely on the assumption of "perfect rationality", which is different from the decision-making behavior of information asymmetry and bounded rationality in the actual market.

[0005] 2. Low computational efficiency and prone to getting trapped in local optima: The model has high dimensionality, and traditional analytical methods are difficult to solve the global equilibrium efficiently, resulting in low computational efficiency.

[0006] 3. Insufficient model completeness and insufficient consideration of the green certificate market: Existing studies mostly focus on the electricity-carbon dual market and have not systematically considered the profound impact of green certificate trading mechanisms on market equilibrium.

[0007] 4. Existing intelligent methods have inherent limitations: multi-agent modeling lacks standardization, and reinforcement learning algorithms are unstable and converge slowly.

[0008] Therefore, there is an urgent need to develop a balanced scheduling method that can take into account the standardization of modeling, the stability of algorithms, and the realism of multi-market coupling, so as to overcome the limitations of existing methods in terms of rational assumptions, computational efficiency, and model integrity, and provide technical support for the coordinated operation and policy optimization of electricity-carbon-green certificates.

[0009] A search revealed no prior art patents that are identical or similar to this invention. Summary of the Invention

[0010] This invention addresses the shortcomings of existing technologies by proposing a market equilibrium scheduling method for electricity, carbon, and green certificates based on multi-agent evolutionary game theory. By introducing a combination of multi-agent evolutionary game theory and deep reinforcement learning, this method effectively overcomes the problems of low computational efficiency, reliance on the assumption of perfect rationality, and susceptibility to local optima in the context of a coupled electricity-carbon-green certificate market. It achieves rapid convergence and global equilibrium scheduling in large-scale complex market environments, significantly improving the returns of market participants and the operational efficiency of the system. This method has strong engineering application value and policy guidance significance.

[0011] The above-mentioned objective of this invention is achieved through the following technical solution: A method for market equilibrium scheduling of electricity-carbon-green certificates based on multi-agent evolutionary game theory includes the following steps: Step 1: Collect historical electricity price and electricity volume data from the electricity market, quota allocation and carbon emission data from the carbon market, and absorption ratio and transaction price data from the green certificate market. Establish a two-layer coupling model of the electricity-carbon-green certificate three markets. This model includes an electricity market clearing model, a carbon emission quota constraint model, and a green certificate ratio constraint model.

[0012] Step 2: Based on the data collected in Step 1 and the established two-layer coupling model of the electricity-carbon-green certificate three-market, multi-agent modeling is carried out, and power generation companies, electricity users, renewable energy companies and regulatory agencies are modeled as multi-agent models; Step 3: Based on the multi-agent model established in Step 2, use the replicating dynamic equation to perform evolutionary modeling of the agent's strategy and construct an evolutionary game model. Step 4: Based on the evolutionary game model constructed in Step 3, introduce the multi-agent proximal policy optimization algorithm MAPPO to construct a multi-agent reinforcement learning model, and train each agent under the framework of "centralized training and distributed execution". Step 5: After training convergence, output the optimal strategy set of each market participant to obtain the equilibrium point of the electricity-carbon-green certificate coupled market.

[0013] Furthermore, the specific steps of step 1 include: (1) Collect historical electricity price and power generation data in the electricity market, carbon emission and quota allocation data in the carbon market, and transaction price and renewable energy power generation data in the green certificate market; (2) Based on the data collected in step (1), establish the electricity market supply and demand balance equation to ensure that market clearing meets the following constraints: in, P i This represents the output of the i-th power generation unit. P j This represents the electricity demand of the j-th user.

[0014] (3) Based on the electricity market supply and demand balance equation in step (2), establish an electricity market clearing model: The basic mechanism of the electricity market is to determine the market-clearing price and electricity volume through the intersection of the supply and demand curves. The cost function of power generation company i on the supply side is denoted as... C i ( q i The utility function of user j on the demand side is: V j ( d j The market equilibrium condition is: in, q i The electricity generated by power generation company i and fed into the grid. d j Electricity consumption of user j The determination of the electricity market clearing price (PE) must meet the following conditions: At this point, the profit function of the power generation company is: The utility of the user is: (4) Based on the data collected in step (1), establish a carbon emission constraint model: In the carbon market, the carbon emissions of each thermal power unit are directly proportional to its power generation: Where γi is the unit's carbon emission factor. The carbon market equilibrium constraint is: Where Ai represents the carbon emission allowance for unit i. The carbon price pc is determined by the Lagrange multipliers of the above constraints. Calculate the carbon emissions and quota differences for each power generation entity: in, α i This represents the unit emission factor of the generator set. A i This indicates the carbon emission allowances obtained by the entity. Q CO2 i represents the carbon trading volume.

[0015] (5) Based on the data collected in step (1), establish a green certificate ratio constraint model. Renewable energy unit k power generation q The number of green certificates issued corresponding to RES k is: Where α is the green certificate conversion coefficient. The green certificate market equilibrium condition is: in, G m The demand for green certificates for businesses or users. Green certificate prices. p g It is determined by the balance of supply and demand.

[0016] The requirement for renewable energy consumption ratio to meet the following conditions is: Where R represents the proportion of renewable energy generation. R min The minimum consumption ratio stipulated by policy, P k Contribute to renewable energy.

[0017] (6) Combining the modeling results of steps (2), (3), (4), and (5), we obtain a two-layer coupling model and constraint set of the three markets of electricity, carbon, and green certificates, which serve as inputs for multi-agent modeling and evolutionary game mechanism construction.

[0018] Furthermore, the specific steps of step 2 include: (1) Based on the two-layer coupling model of the electricity-carbon-green certificate market obtained in step (6) of step 1, an agent model of power generators, users and government regulators is established. The revenue function of each agent is defined as: Where πEi represents electricity market revenue, πCi represents carbon market revenue, and πGi represents green certificate market revenue.

[0019] (2) Based on the agent modeling in step (1) of step 2, the electricity market revenue function is further defined: in, l For electricity price, P i Contributing to the i-th power generator C i ( P i ) is the cost function.

[0020] (3) Based on the agent modeling in step (1) of step 2, the carbon market return function is further defined: in, π c For carbon price, A i For quotas, α i As a carbon emission factor, P i To generate electricity.

[0021] (4) Based on the agent modeling in step (1) of step 2, further define the green certificate market return function: Where, π g For the price of green certificates, G i This refers to the number of green certificates.

[0022] (5) Combining the payoff functions from steps (2), (3), and (4) of step 2, we obtain the total payoff function for each agent, which is then used as the input for the multi-agent evolutionary game: Furthermore, the specific steps of step 3 include: (1) Based on the total payoff function of the multi-agent obtained in step (5) of step 2, the game strategy space is defined to provide a clear set of strategies and feasible boundaries for the evolutionary game model. Each agent selects strategies such as electricity price quotation, carbon trading volume and green certificate trading volume from the strategy set.

[0023] (2) Based on the policy space defined in step (1) of step 3, initialize the policy distribution of various agents and set the individual reward function. f i (x) and the group average return function f (x).

[0024] (3) Based on the reward function set in step (2) of step 3, the policy evolution of the agent is modeled using the replication dynamic equation: in, x i This represents the proportion of the population that chooses strategy i. f i (x) represents the individual payoff for this strategy. This represents the average return for the group.

[0025] (4) Based on the replication dynamic evolution of step (3) in step 3, if the individual return of a certain strategy is higher than the average return of the group, the proportion of that strategy will gradually increase; otherwise, it will gradually decrease. During the update process, the evolutionary stable equilibrium (ESS) criterion is used: in, x ∗ This represents an evolutionarily stable strategy that ensures the stability of the equilibrium point.

[0026] (5) Based on the dynamic update and stability determination in step (4) of step 3, through multiple iterations, the agent strategy gradually converges to the evolutionary stable equilibrium point (ESS) and obtains the equilibrium solution of the electricity-carbon-green certificate coupled market.

[0027] Furthermore, the specific steps of step 4 include: (1) Based on the evolutionary stable equilibrium point obtained in step (5) of step 3, construct a multi-agent reinforcement learning model and set the state space. S Action space A and reward function R The reward function is taken from the total revenue function of each subject. π i .

[0028] (2) Based on the settings in step (1) of step 4, the Multi-Agent Proximal Policy Optimization (MAPPO) method is adopted, and its policy optimization objective function is: in: r t ( i π() represents the policy probability ratio; The dominant function; e This is for truncating parameters.

[0029] (3) Based on the strategy optimization in step (2) of step 4, establish the value function loss: in: R t This represents the actual cumulative return. V ( s t ; i v ) is in the parameter i v The state value function under [condition].

[0030] (4) Based on the results of step (3) in step 4, jointly optimize the parameter updates of the policy network and the value network. The overall optimization objective is: in: c v The loss weights are assigned to the value function. c ent Entropy regularization weights; H ( π ( s t ; i π )) is the entropy function of the strategy.

[0031] (5) Based on the optimization rules in step (4) of step 4, the agent's strategy gradually converges through multiple iterations and updates, and finally obtains the multi-agent equilibrium solution of the three markets of electricity, carbon and green certificates, so as to achieve stable operation and optimal scheduling of the coupled market.

[0032] Furthermore, the specific steps of step 5 include: (1) Based on the multi-agent equilibrium solution obtained in step (5) of step 4, the convergence of the prices and trading volumes of the electricity market, carbon market and green certificate market is monitored, and a tolerance threshold δ is set.

[0033] (2) When the market price difference between any two iterations satisfies: in, p (k) If the market price is represented by the k-th iteration, then the market price is considered to have converged.

[0034] (3) Based on the price convergence determination in step (2) of step 5, perform stability testing on the agent policy distribution. When the rate of change of all agent policy distributions satisfies: in, x (k) i represents the proportion of policy i selected in the k-th iteration, then the policy distribution is considered to be convergent.

[0035] The training convergence condition is: in, d p , d x This is the convergence threshold. When the conditions are met, the output electricity-carbon-green certificate market equilibrium point is: (4) Based on the results of step (3) in step 5, verify the results by combining them with the Evolutionary Stable Equilibrium (ESS) condition: in, x ∗ Let be the policy distribution corresponding to the convergent solution. If the condition is satisfied, then the convergent solution is an evolutionarily stable equilibrium point; (5) Based on the equilibrium determination in step (4) of step 5, the final equilibrium solution of the electricity-carbon-green certificate coupled market is obtained, including market price, trading volume and the optimal strategy distribution of various entities, thereby realizing the stable operation and equilibrium configuration of the coupled market.

[0036] The advantages and beneficial effects of this invention are as follows: 1. This invention systematically addresses the shortcomings of existing models in considering the green certificate market or their superficial coupling by constructing a "two-layer coupling model of the electricity-carbon-green certificate three markets," which includes an electricity market clearing model, a carbon emission quota constraint model, and a green certificate ratio constraint model. Specifically, this technical approach is embodied in step 1 of the specification, which, by collecting data from the three markets and establishing mathematical constraints, endogenizes policy-driven factors such as renewable energy consumption ratios, green certificate trading, and quotas into market equilibrium conditions. This improvement enables the model to realistically reflect the complex operational mechanisms of the interaction between multiple policies and markets, fundamentally enhancing the model's completeness and explanatory power, and providing a complete modeling framework for analyzing the profound impact of green certificate trading on global equilibrium.

[0037] 2. This invention effectively overcomes the reliance on the assumption of perfect rationality in traditional methods by introducing a "multi-agent evolutionary game framework." This technique is specifically embodied in steps 2 and 3 of the specification: First, in step 2, various market participants are modeled as agents with comprehensive payoff functions; then, in step 3, a replicating dynamic equation is used to model the evolution of agent strategies, and evolutionary stable equilibrium (ESS) is used to determine the stability of the equilibrium point. This combined technique can realistically depict the learning, imitation, and adaptation behaviors of market participants under bounded rationality, ensuring that the final equilibrium solution not only has global stability but also good interpretability at the strategy level, avoiding the problem of traditional static equilibrium analysis being disconnected from the actual dynamic evolution process of the market.

[0038] 3. This invention innovatively introduces the "Multi-Agent Proximal Policy Optimization Algorithm (MAPPO)," overcoming the shortcomings of traditional optimization methods, such as low computational efficiency and susceptibility to local optima, as well as the unstable training of existing reinforcement learning methods. This technical approach is specifically embodied in step 4 of the specification: based on the stable policies provided by evolutionary game theory, a multi-agent reinforcement learning model is constructed. Under a framework of centralized training and distributed execution, efficient and stable learning in a high-dimensional complex action space is achieved by jointly optimizing the policy network and the value network. The "dual convergence condition" of price and policy distribution set in step 5 further ensures the reliability of the solution results. This technical approach significantly improves the computational efficiency and convergence success rate of solving large-scale coupled market equilibrium problems, providing a feasible technical tool for real-time market analysis and decision support.

[0039] In summary, the equilibrium scheduling method provided by this invention, through a series of coherent and complementary technical means, forms a complete technical system from modeling and evolution to efficient solution. This method not only possesses higher model realism and algorithm stability, but its output equilibrium results, including electricity prices, carbon prices, green certificate prices, and optimal strategies for each entity, can also provide market operators with accurate price signals and trading volume references. Simultaneously, it provides quantitative basis for government departments to formulate coordinated carbon emission reduction and renewable energy development policies, demonstrating significant engineering application value and policy guidance significance. Attached Figure Description

[0040] Figure 1 This is a flowchart of the training process based on multi-agent evolutionary game theory of the present invention. Detailed Implementation

[0041] The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings: A method for market equilibrium scheduling of electricity-carbon-green certificates based on multi-agent evolutionary game theory, such as Figure 1 As shown, it includes the following steps: Step 1: Collect data and construct a two-layer coupling model of the electricity-carbon-green certificate three-market: Collect historical electricity price and electricity volume data of the electricity market, quota allocation and carbon emission data of the carbon market, and absorption ratio and transaction price data of the green certificate market to establish a two-layer coupling model of the electricity-carbon-green certificate three-market. This model includes an electricity market clearing model, a carbon emission quota constraint model, and a green certificate ratio constraint model.

[0042] The specific steps of step 1 include: (1) Collect historical electricity price and power generation data in the electricity market, carbon emission and quota allocation data in the carbon market, and transaction price and renewable energy power generation data in the green certificate market; (2) Based on the data collected in step (1), establish the electricity market supply and demand balance equation to ensure that market clearing meets the following constraints: in, P i This represents the output of the i-th power generation unit. P j Let G represent the electricity demand of the j-th user, and let D represent the total number of power generation entities and users, respectively.

[0043] (3) Based on the electricity market supply and demand balance equation in step (2), establish an electricity market clearing model: The basic mechanism of the electricity market is to determine the market-clearing price and electricity volume through the intersection of the supply and demand curves. The cost function of power generation company i on the supply side is denoted as... C i ( q i The utility function of user j on the demand side is: V j ( d j The market equilibrium condition is: in, q i The electricity generated by power generation company i and fed into the grid. d j Electricity consumption of user j The determination of the electricity market clearing price (PE) must meet the following conditions: At this point, the profit function of the power generation company for: Utility of demand users for: (4) Based on the data collected in step (1), establish a carbon emission constraint model: In the carbon market, the carbon emissions of each thermal power unit are directly proportional to its power generation: Wherein, γi is the unit's carbon emission factor. This refers to emissions from the carbon market. The carbon market equilibrium constraint is: Where Ai represents the carbon emission allowance for unit i. The carbon price pc is determined by the Lagrange multipliers of the above constraints. Calculate the carbon emissions and quota differences for each power generation entity: in, α i This represents the unit emission factor of the generator set. A i This indicates the carbon emission allowances obtained by the entity. Q CO2 i represents the carbon trading volume.

[0044] (5) Based on the data collected in step (1), establish a green certificate ratio constraint model. Renewable energy unit k power generation q Green certificate issuance volume corresponding to RES k for: Where α is the green certificate conversion coefficient. The green certificate market equilibrium condition is: in, G m The demand for green certificates for businesses or users. Green certificate prices. p g It is determined by the balance of supply and demand.

[0045] The requirement for renewable energy consumption ratio to meet the following conditions is: Where R represents the proportion of renewable energy generation. R min The minimum consumption ratio stipulated by policy, P k Contribute to renewable energy.

[0046] (6) Combining the modeling results of steps (2), (3), (4), and (5), we obtain a two-layer coupling model and constraint set of the three markets of electricity, carbon, and green certificates, which serve as inputs for multi-agent modeling and evolutionary game mechanism construction.

[0047] Step 2: Construct a multi-agent model: Based on the data collected in Step 1 and the established two-layer coupling model of the electricity-carbon-green certificate three-market, construct a multi-agent model, modeling power generation companies, electricity users, renewable energy companies and regulatory agencies as multi-agent models; The specific steps of step 2 include: (1) Based on the two-layer coupling model of the electricity-carbon-green certificate market obtained in step (6) of step 1, establish an agent model for power generators, users, and government regulators. The profit function for each agent... Defined as: Where πEi represents electricity market revenue, πCi represents carbon market revenue, and πGi represents green certificate market revenue.

[0048] (2) Based on the agent modeling in step (1) of step 2, the electricity market revenue function is further defined: in, l For electricity price, P i Contributing to the i-th power generator C i ( P i ) is the cost function.

[0049] (3) Based on the agent modeling in step (1) of step 2, the carbon market return function is further defined: in, π c For carbon price, A i For quotas, α i As a carbon emission factor, P i To generate electricity.

[0050] (4) Based on the agent modeling in step (1) of step 2, further define the green certificate market return function: Where, π g For the price of green certificates, G i This refers to the number of green certificates.

[0051] (5) Combining the profit functions from steps (2), (3), and (4) of step 2, we obtain the total profit function for each subject. And serve as input for multi-agent evolutionary games: Step 3: Construct an evolutionary game mechanism: Based on the multi-agent model established in Step 2, use the replicating dynamic equation to model the evolution of agent strategies and construct an evolutionary game model. The specific steps of step 3 include: (1) Based on the total payoff function of the multi-agent obtained in step (5) of step 2, the game strategy space is defined to provide a clear set of strategies and feasible boundaries for the evolutionary game model. Each agent selects strategies such as electricity price quotation, carbon trading volume and green certificate trading volume from the strategy set.

[0052] (2) Based on the policy space defined in step (1) of step 3, initialize the policy distribution of various agents and set the individual reward function. f i (x) and the group average return function .

[0053] (3) Based on the reward function set in step (2) of step 3, the policy evolution of the agent is modeled using the replication dynamic equation: in, x i This represents the proportion of the population that chooses strategy i. f i (x) represents the individual payoff for this strategy. This represents the average return for the group.

[0054] (4) Based on the replication dynamic evolution of step (3) in step 3, if the individual return of a certain strategy is higher than the average return of the group, the proportion of that strategy will gradually increase; otherwise, it will gradually decrease. During the update process, the evolutionary stable equilibrium (ESS) criterion is used: in, x ∗ This represents an evolutionarily stable strategy that ensures the stability of the equilibrium point.

[0055] (5) Based on the dynamic update and stability determination in step (4) of step 3, through multiple iterations, the agent strategy gradually converges to the evolutionary stable equilibrium point (ESS) and obtains the equilibrium solution of the electricity-carbon-green certificate coupled market.

[0056] Step 4: Based on the evolutionary game model constructed in Step 3, introduce the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm to construct a multi-agent reinforcement learning model, and train each agent under the framework of "centralized training and distributed execution".

[0057] The specific steps of step 4 include: (1) Based on the evolutionary stable equilibrium point obtained in step (5) of step 3, construct a multi-agent reinforcement learning model and set the state space. S Action space A and reward function R The reward function is taken from the total revenue function of each subject. π i .

[0058] (2) Based on the settings in step (1) of step 4, the Multi-Agent Proximal Policy Optimization (MAPPO) method is adopted, and its policy optimization objective function is... for: in: r t ( i π () represents the policy probability ratio; The dominant function; e To truncate parameters, Let be the optimization parameters for the t-th objective.

[0059] (3) Based on the strategy optimization in step (2) of step 4, establish the value function loss. : in: R t This represents the actual cumulative return. V ( s t ; i v ) is in the parameter i v The state value function under [condition].

[0060] (4) Based on the results of step (3) in step 4, jointly optimize the parameter updates of the policy network and the value network, and optimize the overall objective. for: in: c v The loss weights are assigned to the value function. c ent Entropy regularization weights; H ( π ( s t ; i π )) is the entropy function of the strategy.

[0061] (5) Based on the optimization rules in step (4) of step 4, the agent's strategy gradually converges through multiple iterations and updates, and finally obtains the multi-agent equilibrium solution of the three markets of electricity, carbon and green certificates, so as to achieve stable operation and optimal scheduling of the coupled market.

[0062] Step 5: After training convergence, output the optimal strategy set of each market participant to obtain the equilibrium point of the electricity-carbon-green certificate coupled market, including electricity price, carbon trading price and green certificate price, as well as the trading strategy and profit level of each participant.

[0063] The specific steps of step 5 include: (1) Based on the multi-agent equilibrium solution obtained in step (5) of step 4, the convergence of the prices and trading volumes of the electricity market, carbon market and green certificate market is monitored, and a tolerance threshold δ is set.

[0064] (2) When the market price difference between any two iterations satisfies: in, p (k) If the market price is represented by the k-th iteration, then the market price is considered to have converged.

[0065] (3) Based on the price convergence determination in step (2) of step 5, perform stability testing on the agent policy distribution. When the rate of change of all agent policy distributions satisfies: in, x (k) i represents the proportion of policy i selected in the k-th iteration, then the policy distribution is considered to be convergent.

[0066] The training convergence condition is: in, d p , d x This is the convergence threshold. When the conditions are met, the output electricity-carbon-green certificate market equilibrium point is: (4) Based on the results of step (3) in step 5, verify the results by combining them with the Evolutionary Stable Equilibrium (ESS) condition: in, x ∗ Let be the policy distribution corresponding to the convergent solution. If the condition is satisfied, then the convergent solution is an evolutionarily stable equilibrium point; (5) Based on the equilibrium determination in step (4) of step 5, the final equilibrium solution of the electricity-carbon-green certificate coupled market is obtained, including market price, trading volume and the optimal strategy distribution of various entities, thereby realizing the stable operation and balanced configuration of the coupled market.

[0067] The innovation of this invention lies in: This invention addresses the problems in the prior art mentioned in the background section and provides specific solutions: 1. Rational assumptions are unrealistic. Traditional methods rely on the assumption of "perfect rationality," which differs from the decision-making behavior in the actual market characterized by information asymmetry and bounded rationality.

[0068] A "multi-agent evolutionary game" framework is introduced.

[0069] By replicating dynamic equations to simulate the evolution of agent policies, the agent is not required to make optimal decisions at the beginning. Instead, it is gradually optimized through learning and imitation (policies with above-average returns are adopted more often).

[0070] 2. Low computational efficiency, prone to getting trapped in local optima. The model has high dimensionality, making it difficult for traditional analytical methods to efficiently solve for global equilibrium, resulting in low computational efficiency.

[0071] Introducing the "Multi-Agent Proximal Policy Optimization Algorithm (MAPPO)". • It utilizes deep reinforcement learning within a "centralized training, distributed execution" framework to handle high-dimensional state and action spaces. • By jointly optimizing the policy network and value network, it achieves efficient and stable learning, significantly improving the computational efficiency and convergence reliability of equilibrium solutions, and avoiding the problem of traditional optimization methods getting trapped in local optima.

[0072] 3. Insufficient model completeness and insufficient consideration of the green certificate market. Existing research focuses mainly on the electricity-carbon dual market and has not systematically considered the profound impact of green certificate trading mechanisms on market equilibrium.

[0073] A two-layer coupling model of the electricity-carbon-green certificate three-market structure is constructed. • In step 1, a green certificate ratio constraint model is established, incorporating the issuance and trading of green certificates and renewable energy consumption ratio policies into market equilibrium conditions. • In the multi-agent revenue function of step 2, a "green certificate market revenue" is included, making the behavior of renewable energy companies and green certificate purchasers an intrinsic driving factor of the model.

[0074] 4. Existing intelligent methods have inherent defects. Multi-agent modeling lacks standardization, and reinforcement learning algorithms are unstable and converge slowly.

[0075] It provides a standardized modeling process and a stable combination of algorithms. • From data collection → coupled model construction → agent definition → evolutionary game theory → MAPPO training, a standardized modeling process is formed, enhancing reproducibility. • The MAPPO algorithm was chosen because its proximal pruning mechanism is designed to improve training stability, and combined with the dual convergence conditions of price and policy distribution, the reliability of the results is ensured.

[0076] Example 1: Equilibrium Solution of the Electricity-Carbon-Green Certificate Coupled Market To verify the effectiveness of the method of this invention, a simple electricity-carbon-green certificate coupling market example was constructed, comprising one thermal power unit, one wind power unit, and one user. Relevant data is as follows: Cost function for thermal power units: carbon emission factors t / MWh, quota t; The cost of the wind turbine is 0, and the green certificate conversion factor is [missing information]. ; User utility function: ; Minimum renewable energy consumption ratio: .

[0077] (1) The initial clearing result of the electricity market is obtained based on the supply and demand functions: Clearing electricity price: Yuan / MWh; Thermal power output: MWh; Wind power output: MWh; User electricity consumption: MWh.

[0078] (2) The carbon market balances the emissions from thermal power plants. Compared to quotas t, remaining quota 2t. The carbon price is determined through supply and demand balance: Yuan / t.

[0079] (3) The issuance volume of green certificates for wind power in the green certificate market is: Zhang. The proportion of renewable energy required by user demand: open.

[0080] The green certificate market clearing process yielded the following results: Green Certificate Price: Yuan / sheet.

[0081] (4) After policy evolution, the strategies of each agent in the multi-agent evolutionary game converge to: Thermal power units opted for a "medium-priced offer + sale of remaining quotas"; The wind turbine generators opted for a "full grid connection + sale of surplus green certificates" approach. Users can choose "Normal electricity consumption + purchase green certificates as needed".

[0082] (5) The final equilibrium of MAPPO reinforcement learning is near the initial equilibrium provided by the replication dynamics. Through multi-agent reinforcement learning, it further approximates the global equilibrium, and the final convergence result is: Electricity price: Yuan / MWh Carbon price: Yuan / t Green Certificate Price: Yuan / sheet Thermal power output: 59 MWh; Wind power output: 21 MWh User requirement: 80 MWh All three main strategies remained stable, satisfying the ESS condition.

[0083] This example demonstrates that the method of the present invention can quickly find the equilibrium solution of electricity-carbon-green certificates in a simplified scenario, verifying the feasibility and effectiveness of the proposed method.

[0084] It should be emphasized that the embodiments described in this invention are illustrative rather than limiting. Therefore, this invention includes, but is not limited to, the embodiments described in the specific implementation. Any other implementations derived by those skilled in the art based on the technical solutions of this invention are also within the scope of protection of this invention.

Claims

1. A multi-agent evolutionary game-based electricity-carbon-green certificate market equilibrium scheduling method, characterized in that: The method comprises the following steps: Step 1, collect historical electricity price and power data of the electricity market, quota allocation and carbon emission data of the carbon market, and consumption ratio and transaction price data of the green certificate market, and establish a double-layer coupled model of the electricity-carbon-green certificate three markets, which comprises an electricity market clearing model, a carbon emission quota constraint model and a green certificate ratio constraint model; Step 2, based on the data collected in step 1 and the double-layer coupled model of the electricity-carbon-green certificate three markets established, multi-agent modeling is performed, and power generation enterprises, power users, renewable energy enterprises and regulatory agencies are modeled as multi-agent models; Step 3, based on the multi-agent model established in step 2, a replicator dynamics equation is used to model the evolution of the agent strategy, and an evolutionary game model is constructed; Step 4, based on the evolutionary game model constructed in step 3, a multi-agent proximal policy optimization algorithm MAPPO is introduced to construct a multi-agent reinforcement learning model, and each agent is trained under the "centralized training, distributed execution" framework; Step 5, after training convergence, the optimal strategy set of each market subject is output, and the equilibrium point of the electricity-carbon-green certificate coupled market is obtained.

2. The method of claim 1, wherein the method is characterized in that: The specific steps of step 1 comprise: (1) collect historical electricity price and power data of the electricity market, carbon emission and quota allocation data of the carbon market, and transaction price and renewable power generation data of the green certificate market; (2) based on the data collected in step (1), an electricity market supply and demand balance equation is established to ensure that the market clearing meets the following constraints: ; wherein, P i represents the power output of the i-th power generation subject, P j represents the power demand of the j-th user; (3) according to the electricity market supply and demand balance equation in step (2), an electricity market clearing model is established: The basic mechanism of electricity market is to determine the market clearing price and electricity quantity by the intersection of supply curve and demand curve; the cost function of supply side power generation enterprise i is denoted as C i ( q i ), and the utility function of demand side user j is V j ( d j ); the market equilibrium condition is: ; wherein, q i is the on-grid power of the power generation enterprise i, d j is the power consumption of the user j The determination of the electricity market clearing price pe satisfies the following conditions: ; At this time, the profit function of the power generation enterprise is: ; The utility of the demand user is: ; (4) based on the data collected in step (1), a carbon emission constraint model is established: In the carbon market, the carbon emission of each thermal power unit is proportional to the power generation: ; Where γi is the carbon emission factor of the unit; the carbon market balance constraint is: ; Where Ai is the carbon emission quota of unit i; the carbon price pc is determined by the Lagrange multiplier of the above constraint The carbon emission and quota difference of each power generation subject is calculated: ; wherein, α i represents the unit emission factor of the generator set, A i represents the carbon emission quota obtained by the subject, Q CO2 i represents the carbon trading amount; (5) based on the data collected in step (1), a green certificate ratio constraint model is established, Renewable energy unit k power generation q The green certificate issuance amount corresponding to RES k is: ; Where α is the green certificate conversion coefficient; the green certificate market balance condition is: ; Wherein, G m The demand for green certificates by enterprises or users; the price of green certificates p g Determined by supply and demand balance; The renewable energy consumption ratio satisfies: ; wherein R is the renewable energy generation proportion, R min is the minimum consumption proportion prescribed by policy, P k is the renewable energy output; (6) the modeling results of steps (2), (3), (4) and (5) are combined to obtain a double-layer coupled model of the electricity-carbon-green certificate three markets and a constraint set, which is used as the input of the multi-agent modeling and evolutionary game mechanism construction.

3. The method of claim 1, wherein the method is characterized in that: The specific steps of step 2 comprise: (1) based on the double-layer coupled model of the electricity-carbon-green certificate market obtained in step 1 (6), an agent model of the power supplier, the user and the government regulatory party is established; the benefit function of each agent is defined as: ; Where πE i is the electricity market benefit, πC i is the carbon market benefit, and πG i is the green certificate market benefit; (2) based on the agent modeling in step 2 (1), the electricity market benefit function is further defined: ; wherein, λ is the electricity price, P i is the output of the i-th power producer, C i ( P i is the cost function; (3) Based on the agent modeling of step 2 (1), further define the carbon market revenue function: ; wherein, π c is the carbon price, A i is the quota, α i is the carbon emission factor, P i is the power output; (4) Based on the agent modeling of step 2 (1), further define the green certificate market revenue function: ; wherein π g is the green certificate price, G i is the green certificate quantity; (5) Integrate the revenue functions of steps 2 (2), (3) and (4) to obtain the total revenue function of each subject, and use it as the input of the multi-agent evolutionary game: 。 4. The method of claim 1, wherein the method is characterized in that: The specific steps of step 3 include: (1) Based on the multi-agent total revenue function obtained in step 2 (5), define the game strategy space to provide a clear strategy set and feasible boundary for the evolutionary game model; each agent selects strategies such as power price, carbon trading volume and green certificate trading volume in the strategy set; (2) Initialize the strategy distribution of each type of agent based on the strategy space defined in step 3 (1), and set the individual profit function f i (x) and the group average profit function f (x); (3) Based on the revenue function set in step 3 (2), use the replicator dynamic equation to model the strategy evolution of the agent: ; wherein, x i pi represents the proportion of the population that chooses the ith strategy, f i (x) is the individual payoff for that strategy, is the population average payoff; (4) Based on the replicator dynamic evolution of step 3 (3), if the individual revenue of a strategy is higher than the average revenue of the group, the proportion of the strategy gradually increases, otherwise it gradually decreases; In the updating process, the evolutionary stable equilibrium (ESS) criterion is used: ; wherein x ∗ denotes an evolutionary stable strategy, ensuring stability of the equilibrium point; (5) Based on the dynamic updating and stability judgment of step 3 (4), through multiple iterations, the agent strategy gradually converges to the evolutionary stable equilibrium point (ESS), and the equilibrium solution of the power-carbon-green certificate coupled market is obtained.

5. The method of claim 1, wherein the method is characterized in that: The specific steps of step 4 include: (1) Based on the evolutionary stable equilibrium point obtained in step 3, step (5), a multi-agent reinforcement learning model is constructed, and the state space S , action space A , and reward function R are set, wherein the reward function is taken from the total income function of each agent π i ; (2) Based on the setting of step 4 (1), use the multi-agent proximal policy optimization (MAPPO) method, and the strategy optimization objective function is: ; wherein: r t (3) Based on the strategy optimization of step 4 (2), establish the value function loss: π is a policy probability ratio; is a advantage function; (4) Based on the results of step 4 (3), jointly optimize the parameter update of the strategy network and the value network, and the overall optimization objective is: is a truncation parameter;​ (5) Based on the optimization rule of step 4 (4), through multiple iterative updates, the strategy of the agent gradually converges, and finally the multi-agent equilibrium solution of the power-carbon-green certificate market is obtained, realizing the stable operation and optimal scheduling of the coupled market. ; where: R t is the actual cumulative return; V s t ; The specific steps of step 5 include: v is the state value function under parameters (1) Based on the multi-agent equilibrium solution obtained in step 4 (5), monitor the convergence of the prices and trading volumes of the power market, carbon market and green certificate market, and set a tolerance threshold δ; v .​ (2) When the market price difference between any two iterations satisfies: ; where: c v is a value function loss weight; c ent is an entropy regularization weight; H π s t (3) Based on the price convergence judgment of step 5 (2), detect the stability of the agent strategy distribution; when the strategy distribution change rate of all subjects satisfies: π is an entropy function of the policy;​​​ The training convergence condition is:

6. The method of claim 1, wherein the method is based on multi-agent evolutionary game. (4) Based on the results of step 5 (3), verify the evolutionary stable equilibrium (ESS) condition: (5) Based on the equilibrium judgment of step 5 (4), the final equilibrium solution of the electricity-carbon-green certificate coupled market is obtained, including market prices, trading volumes and optimal strategy distribution of various subjects, thus realizing the stable operation and equilibrium configuration of the coupled market. ​ ; wherein, p (k) Let Pkdenote the market price at the kth iteration, then the market price is considered to converge if ​ ; wherein, x (k) i denotes the proportion of strategy i selected in the kth iteration, then the strategy distribution is considered to converge; ​ ; wherein, ​ p , ​ x is a convergence threshold; when the condition is met, the output power-carbon-green certificate market equilibrium point: ; ​ ; wherein, x ∗ is the corresponding strategy distribution of the convergence solution; if the condition is satisfied, the convergence solution is an evolutionary stable equilibrium point; ​

Images