Multi-agent bidding behavior simulation method and device based on non-independent opponent modeling

Abstract

The invention provides a multi-subject bidding behavior simulation method and device based on non-independent opponent modeling. The method comprises the following steps: loading a pre-constructed fitting function in an SOM-based multi-subject market framework; for each market subject of the electricity market, when the market subject makes each round of decision, a fitting function speculation mode is started to infer action probability distribution of other market subjects and estimate a potential market clearing result; starting an action mode of a fitting function to determine the quotation quantity of the market subject according to the potential market clearing result; and calculating a real market clearing result by using a preset electricity market clearing mechanism and the quantity report quoted price of all the market subjects. According to the scheme, possible bidding of multiple opponents is taken into consideration in the strategy of the market subject, and electricity market transaction is simulated from bottom to top under behavior driving, so that a more competitive quotation strategy is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a multi-agent bidding behavior simulation method and device based on non-independent adversary modeling. BACKGROUND

[0002] Traditional power market transaction clearing models usually use optimization theory for modeling, and formulate bidding strategies (or bidding strategies) by minimizing cost or maximizing profit.

[0003] However, in real decision-making, there are different preferences and heterogeneity of individuals, resulting in differences in decision-making behavior. Only through traditional mathematical or economic algorithm modeling, it is difficult to capture the influence of individual heterogeneity and uncertainty, so that the competitiveness of the formulated bidding strategy is poor. SUMMARY

[0004] Therefore, the embodiments of the present application provide a multi-agent bidding behavior simulation method and device based on non-independent adversary modeling to obtain a more competitive bidding strategy.

[0005] To achieve the above-mentioned purpose, the embodiments of the present application provide the following technical solutions:

[0006] The first aspect of the embodiments of the present application discloses a multi-agent bidding behavior simulation method based on non-independent adversary modeling, which comprises:

[0007] loading a fitting function in a SOM multi-agent market framework constructed in advance;

[0008] For each market agent of the power market, when each round of decision-making of the market agent is performed, the inference mode of the fitting function is enabled to infer the action probability distribution of other market agents and estimate the potential market clearing result;

[0009] The action mode of the fitting function is enabled to determine the bidding volume of the market agent according to the potential market clearing result, and the inference mode and the action mode of the fitting function share a set of preset parameters;

[0010] The real market clearing result is calculated by using the preset power market clearing mechanism and the bidding volume of all the market agents.

[0011] Preferably, for each market agent of the power market, when each round of decision-making of the market agent is performed, the inference mode of the fitting function is enabled to infer the action probability distribution of other market agents and estimate the potential market clearing result, comprising:

[0012] For each market agent of the power market, when each round of decision-making of the market agent is performed, the inference mode of the fitting function is enabled;

[0013] acquire real historical market clearing results, market states, and characteristic attributes of other market participants;

[0014] input the real historical market clearing results, the market states, and the characteristic attributes of the other market participants into the fitting function enabled with the speculation mode to infer the bidding and offering strategy of the other market participants;

[0015] estimate potential market clearing results by using a preset power market clearing mechanism and the bidding and offering strategy of the other market participants.

[0016] Preferably, the action mode of the fitting function is enabled to determine the bidding and offering of the market participant according to the potential market clearing results, including:

[0017] enabling the action mode of the fitting function;

[0018] acquiring market states and characteristic attributes of the market participant;

[0019] inputting the potential market clearing results, the market states, and the characteristic attributes of the market participant into the fitting function enabled with the action mode to infer the bidding and offering of the market participant, including a bidding and offering strategy and a value of the bidding and offering strategy.

[0020] Preferably, the fitting function is obtained by reinforcement learning training of an Actor-Critic model, and the Actor-Critic model is composed of a critic network and a policy network.

[0021] Preferably, after the real market clearing results are calculated, the method further includes:

[0022] updating the preset parameters shared by the speculation mode and the action mode of the fitting function according to the real market clearing results.

[0023] The second aspect of the embodiment of the application discloses a multi-agent bidding behavior simulation device based on non-independent opponent modeling, and the device includes:

[0024] a loading unit configured to load a fitting function in a pre-constructed SOM multi-agent market framework;

[0025] a first enabling unit configured to, for each market participant in a power market, enable a speculation mode of the fitting function to infer an action probability distribution of other market participants and estimate potential market clearing results when each round of decision-making of the market participant is performed;

[0026] a second enabling unit configured to enable an action mode of the fitting function to determine the bidding strategy of the market subject according to the potential market clearing result, the speculation mode and the action mode of the fitting function sharing a set of preset parameters;

[0027] a calculation unit configured to calculate a real market clearing result by using a preset power market clearing mechanism and the bidding strategy of all the market subjects.

[0028] Preferably, the first enabling unit comprises:

[0029] an enabling module configured to enable the speculation mode of the fitting function for each market subject of the power market when the market subject makes a decision in each round;

[0030] an acquisition module configured to acquire a real historical market clearing result, a market state, and a characteristic attribute of another market subject;

[0031] an input module configured to input the real historical market clearing result, the market state, and the characteristic attribute of the another market subject into the fitting function enabled with the speculation mode to infer a bidding strategy of the another market subject;

[0032] an estimation module configured to estimate a potential market clearing result by using a preset power market clearing mechanism and the bidding strategy of the another market subject.

[0033] Preferably, the second enabling unit comprises:

[0034] an enabling module configured to enable the action mode of the fitting function;

[0035] an acquisition module configured to acquire a market state and a characteristic attribute of the market subject;

[0036] an input module configured to input the potential market clearing result, the market state, and the characteristic attribute of the market subject into the fitting function enabled with the action mode to infer a bidding strategy of the market subject, the bidding strategy of the market subject comprising a bidding strategy and a value of the bidding strategy.

[0037] Preferably, the fitting function is obtained by reinforcement learning training of an Actor-Critic model, the Actor-Critic model being composed of a critic network and a policy network.

[0038] Preferably, the apparatus further comprises:

[0039] an updating unit configured to update the preset parameters shared by the speculation mode and the action mode of the fitting function according to the real market clearing result.

[0040] Based on the above-mentioned embodiment of the application, a multi-agent bidding behavior simulation method and device based on non-independent opponent modeling are provided. The method is as follows: a fitting function in a SOM-based multi-agent market framework is loaded; for each market agent of a power market, when each round of decision-making of the market agent is performed, a speculation mode of the fitting function is enabled to infer the action probability distribution of other market agents and estimate a potential market clearing result; an action mode of the fitting function is enabled to determine the bid quantity and bid price of the market agent according to the potential market clearing result; and a preset power market clearing mechanism and the bid quantity and bid price of all market agents are used to calculate a real market clearing result. In the strategy of the market agent, the possible bids of multiple opponents are considered, and the power market transaction is simulated from bottom to top by behavior driving, so that a more competitive bid strategy is obtained. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only constitute the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0042] Figure 1 The power spot market structure provided for the embodiment of the present application;

[0043] Figure 2 The price formation mechanism of marginal price clearing provided for the embodiment of the present application;

[0044] Figure 3 The unified price clearing flow framework diagram provided for the embodiment of the present application;

[0045] Figure 4 The flowchart of the multi-agent bidding behavior simulation method based on non-independent opponent modeling provided for the embodiment of the present application;

[0046] Figure 5 The decision inference and decision framework based on SOM provided for the embodiment of the present application;

[0047] Figure 6 The optimization training framework based on Actor-Critic provided for the embodiment of the present application;

[0048] Figure 7 The full-province load demand example diagram of SX province and SD province provided for the embodiment of the present application;

[0049] Figure 8 The example diagram of the coal purchasing price index provided for the embodiment of the present application;

[0050] Figure 9 An example graph of the relationship between the power market clearing price and the market structure for embodiments of the present application;

[0051] Figure 10 An example graph of the relationship between the power market clearing price and the market structure for embodiments of the present application;

[0052] Figure 11 An example graph of the relationship between the power market clearing price and the market structure for embodiments of the present application;

[0053] Figure 12 An example graph of the relationship between the power market clearing price and the market structure for embodiments of the present application;

[0054] Figure 13 An example graph of the relationship between the power market clearing price and the market structure for embodiments of the present application;

[0055] Figure 14 An example graph of the relationship between the power market clearing price and the market structure for embodiments of the present application;

[0056] Figure 15 An example graph of the relationship between the power market clearing price and the market structure for embodiments of the present application;

[0057] Figure 16 An example graph of the relationship between the power market clearing price and the market structure for embodiments of the present application;

[0058] Figure 17 An example graph of the relationship between the power market clearing price and the market structure for embodiments of the present application; DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative work fall within the protection scope of the present application.

[0060] In this application, the terms "comprise", "contain", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements not only includes those elements, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0061] As the marketization of the power industry continues to deepen, the investment and production decision-making behavior of power generation enterprises gradually marketizes, which may lead to an increase in the supply security risk in the green transformation process of the power sector. After the pilot provinces and regions in the new round of power marketization entered the trial operation stage, they began to promote new initiatives for various types of energy green transformation under the conditions of the power market. Under the "double carbon" goal, various energy transformation strategies, such as carbon emissions trading markets, renewable energy quota systems, and renewable energy on-grid electricity price subsidies, will incentivize investment in various non-fossil energy power generation construction. Under the influence of the priority effect of renewable energy and the progress effect of green technology, competitive power markets will increase the probability of stranded costs for thermal power generation. At the same time, under the influence of the double factors of declining profits in the thermal power industry and the great uncertainty of renewable energy output, the risk of power supply security may be expanded. Based on this, the present scheme adopts the method of agent-based modeling (ABM) combined with the multi-agent non-independent adversary modeling framework to build a power spot market bidding behavior simulation algorithm to analyze the market-oriented production decisions of power generation enterprises.

[0062] It is found in the research that the traditional power market transaction clearing model usually uses optimization theory for modeling, and formulates the bidding strategy (or bidding strategy) by minimizing the cost or maximizing the profit.

[0063] On the one hand, in some complex trading scenarios, the objective function of the optimization model is nonlinear, which causes difficulties in optimization solution. On the other hand, there are individual differences in decision-making behavior due to different preferences and heterogeneity in real decision-making. In addition, most market participants are bounded rational and cannot have complete information, which makes the trading and clearing of the power market have great uncertainty. It is difficult to capture the influence of individual heterogeneity and uncertainty through traditional mathematical or economic algorithm modeling.

[0064] To this end, for the limited rationality of market subjects and incomplete information transactions, the ABM is used to model the power market clearing. The ABM can realize the incomplete information simulation of the limited rationality, and can better emphasize the importance of the heterogeneity, two-way interaction of the power system, and the feedback between the behavior subjects and the institutions. Meanwhile, in order to further adjust and optimize the limited rationality of the subjects, the research intends to help the ABM subjects find a balance between "zero intelligence" and complete rationality through reinforcement learning. Although reinforcement learning can be well combined with ABM due to its own theoretical framework characteristics, the limitations of traditional reinforcement learning algorithms such as low-dimensional and discrete state space and action space still need further research.

[0065] The scheme builds a non-independent decision mechanism between subjects based on the ABM framework, and proposes a multi-agent bidding behavior simulation algorithm based on self-other modeling (SOM). The algorithm not only retains the description of the heterogeneity of the subjects and the interaction with the environment in the ABM, but also considers the possible bids of the opponents in the strategy of the subjects, to drive the simulation of the power market transactions from bottom to top. The model uses a deep neural network to fit the reasoning function, and uses the Actor-Critic reinforcement learning algorithm for policy iteration and value iteration to realize the decision optimization of each market subject.

[0066] The scheme has the following characteristics: first, the limited rationality characteristics of market participants (referred to as market subjects or subjects) are fully described. The multi-agent bidding behavior simulation algorithm based on SOM optimizes the strategy of each subject according to the optimization of the future possible income of the subject; second, the non-independent decision between subjects is innovatively modeled. The strategy of the subject considers the possible bids of the opponents, and estimates the future potential income of itself; third, the incomplete information bidding in the market is more in line with the reality. The bidding strategy of the subject can only be based on the cost, historical income, and potential market demand that can be obtained by the subject.

[0067] The power market can be regarded as a complex system and simulated by ABM. In the simulation, coal-fired power plants can be regarded as behavior subjects (agents, i.e. market subjects). These behavior subjects have the following heterogeneity:

[0068] On the one hand, the attribute heterogeneity includes the preferences of coal-fired power plants for risk, technology, and their own resource endowments; on the other hand, the strategy heterogeneity is that different coal-fired power plants can learn to find new bidding strategies.

[0069] These heterogeneous entities, based on their own attributes and strategies, make different bidding decisions in the day-ahead and real-time markets. The interaction between the entities and the environment is bidirectional. Not only does each coal-fired power plant bid based on current market demand, historical market clearing patterns, and its own preferences, forming the clearing prices in the day-ahead and real-time markets, but it also influences the market's influence on the entities' bidding, forming a new market-clearing price. This market-clearing price, along with the new market demand, is then transmitted to the coal-fired power plant, which then iterates and updates its bidding strategy based on its actual profits.

[0070] It should be noted that the implementation of this plan will involve the electricity market clearing mechanism and the modeling of market entities. In order to better understand the content of this plan, the principles of the two parts of "electricity market clearing mechanism" and "market entity modeling" are first explained here.

[0071] 1. Explanation of the electricity market clearing mechanism:

[0072] In the electricity spot market, the market is mainly divided into the day-ahead market and the real-time market. Figure 1 The electricity spot market structure shown in the figure, for electricity spot market transactions on day t, the first stage is to trade in the day-ahead market on day t-1. In the day-ahead market, each market participant has a given predicted electricity market demand D on day t. DA,t Under this condition, according to its own marginal power generation cost MC and installed capacity Make quotations and quantities, and obtain day-ahead market settlement through market transaction clearing.

[0073] The second stage occurs on day t, which is called the real-time market. Due to many factors such as demand forecast deviation, volatility of actual demand, uncertainty of renewable energy output, etc., the actual electricity consumption on that day is different from the planned D DA,t There is a gap. This electricity demand gap is traded in the real-time market. In the real-time market, the actual electricity market demand deviation D of each market participant on day t is RT,t Under this situation, according to its own marginal power generation cost and remaining installed capacity Compared to the day-ahead market, the real-time market fills the gap between power system forecasts and actual operational demand, reflecting the power system's ultra-short-term power resource scarcity.

[0074] For a trading market, the goal is to minimize the market cost while meeting the market electricity demand. Therefore, the problem can be expressed as formula (1).

[0075] (1);

[0076] In formula (1), N is the set of market actors participating in the electricity market bidding (referred to as market actors for short), is the market trading period, P t is the actual market settlement price on day t, and q i,t is the actual trading volume of market entity i on day t, is the installed capacity of the market entity (representing the upper limit of its power generation), D t is the load demand of the market, and the actual transaction volume of all market players on that day must meet the load demand, that is, .

[0077] The market clearing price is settled according to the marginal clearing price (Uniform Pricing, UP). Market managers calculate the marginal clearing price based on the bids of all market participants and then settle the transactions for each successful market participant at this UP. This clearing method is a practical and feasible way to minimize electricity market transaction costs, achieving the goal of optimizing market transaction costs and meeting market electricity demand within a single day.

[0078] The price formation mechanism of marginal price clearing is as follows Figure 2 As shown, Figure 2 In the figure, P is the clearing price, and Q is the corresponding production volume under the power market demand D. The bids and quantities of various market participants collectively form a stepped bid curve. Assuming the exogenous market demand is D (red dashed line), when the bids of market participants, cumulatively ranked from lowest to highest, meet the market demand, the bid of the last market participant that meets market demand D becomes the current market clearing settlement price. At this point, the participants in the white area successfully trade. With the exception of the marginal participants, all trades are completed based on their actual bids and the unified market marginal price. The marginal participants trade at the market's unified marginal price, based on the difference between the market demand and the bids of other participants. Conversely, the participants in the gray shaded area fail to trade.

[0079] The clearing mechanism of electricity markets at all levels is as follows: Figure 3 As shown, first, market entities obtain market load demand Whether in the day-ahead market or the real-time market, market players bid for each hour, so the market load demand is a 24-dimensional vector. Then, n market players bid for the market load demand, forming a quotation and quantity sequence . The market is based on the quote " (1≦i≦n)" Sort from low to high to form a new sequence ,satisfy The resulting sequence corresponds to Figure 2 The step-like quotation curve in the series is used. The quotation quantity is accumulated from this sequence until the market load demand D is met.t and forms the marginal price P t Each market participant is divided into non-marginal clearing participants S non-marginal , marginal clearing participants S marginal , and non-clearing participants S fail The transaction price is shown in formula (2).

[0080] (2);

[0081] In formula (2), is the actual transaction price of market participant i after clearing at the market marginal price, and in the subsequent calculation of the market participant's income or return (as mentioned in formula (10) below). Accordingly, the clearing capacity of the market participant is shown in formula (3).

[0082] (3);

[0083] In formula (3), is the actual transaction volume of market participant i after clearing at the market marginal price, and the subscripts i and j represent market participants (only to distinguish between formulas).

[0084] It should be noted that for Figure 3 , formula (2) and formula (3), since the marginal price of the entire market is the market clearing price, the marginal price of the market here is represented as P t , t represents time, so P t represents the price at time t;

[0085] Specifically, the actual transaction price of market participant i at time t is shown in formula (2), and under the condition of uniform pricing, it can be calculated as , formula (2) indicates that only when market participant i can clear =P t , otherwise market participant i has no transaction (price is 0);

[0086] Figure 3 The general process of uniform price clearing is shown, so Figure 3 each parameter in the formula does not have a "t" subscript.

[0087] For coal-fired power plants, the generation cost can be divided into fuel cost and other fixed cost according to whether it is related to the generation capacity.

[0088] Among them, fuel costs are related to the actual amount of electricity produced, losses and the trading price of power coal, while other fixed costs include equipment maintenance, power plant operation and other costs, which can be regarded as fixed constants that are unrelated to the actual amount of electricity produced.

[0089] Therefore, the production cost of coal-fired power plant i (1≦i≦n) is set as shown in formula (4).

[0090] (4);

[0091] In formula (4), represents the total production cost of coal-fired power plant i on day t, represents the amount of electricity that coal-fired power plant i participates in the electricity market bidding on day t, represents the self-use rate of power generation of coal-fired power plant i, so “ / 1- ” represents the amount of electricity produced by the power plant after correction;

[0092] represents the coal consumption coefficient of coal-fired power plant i, The higher the value, the more coal is consumed to produce each unit of electricity, and the lower the efficiency of the power plant; represents the thermal coal transaction price factor (related to time t);

[0093] Considering that the impact of thermal coal transaction prices on production decisions has a lag, General lag and average lag were designed for verification; represents other fixed costs, which is a constant term that varies with time t and power generation. Not relevant.

[0094] Under perfectly competitive market conditions, power generation companies generally bid according to their marginal power generation costs. According to formula (4), we can calculate the power generation cost with respect to power generation. The partial derivative of is used to obtain the marginal power generation cost function of coal-fired power plant i as shown in formula (5).

[0095] (5);

[0096] In formula (5), represents the marginal production cost of coal-fired power plant i on day t. At this time, the bid price of coal-fired power plant i can be expressed as formula (6).

[0097] (6);

[0098] In formula (6), represents the bidding strategy coefficient of coal-fired power plant i, satisfying 1≦ ≦ By equation (6), the marginal cost of coal-fired power plant i can be expressed as times of the marginal cost of coal-fired power plant i, while depends on the preference of coal-fired power plant i itself and the current market demand.

[0099] According to the generation cost of equation (4), the bidding strategy of coal-fired power plant i in the electricity market can be expressed as maximizing its own profit under the constraint of the installed capacity of the power plant; as shown in equation (7).

[0100] (7);

[0101] where P t is the market marginal clearing price.

[0102] The above is an explanation of the electricity market clearing mechanism, which is mainly composed of the contents of equations (1) to (3), etc.

[0103] 2. Explanation of market subject modeling:

[0104] In order to better deconstruct the strategies of various market subjects in the electricity market and reveal how micro-subject behaviors accumulate to emerge macro-observations, this scheme adopts reinforcement learning (RL) to learn the ABM-based electricity market clearing simulation process. Therefore, it is necessary to define the state, action and reward function of the electricity market subject bidding. The state space is determined by the electricity market environment, and the state s(t) of the t-day market is composed of the day-ahead market and the real-time market demand, as shown in equation (8).

[0105] (8);

[0106] The bidding action of coal-fired power plant i is represented as a combination vector of a bid price and a bid quantity, as shown in equation (9).

[0107] (9);

[0108] In fact, compared with directly using the bid price of coal-fired power plant , the bid strategy coefficient represents bidding at several times the marginal cost, which can better represent the preference of the coal-fired power plant bidding strategy.

[0109] When ​​= 1 means that the coal-fired power plant i quotes according to its marginal cost completely, in order to get higher market competitiveness in the way of reducing profit. By combining the above formula (6), the real quote of the coal-fired power plant can be further calculated. The return function of the coal-fired power plant i is shown in formula (10).

[0110] (10);

[0111] In formula (10), is the return of the coal-fired power plant i in the market clearing on day t; it can be seen that the future cumulative return of the coal-fired power plant i in period T is shown in formula (11).

[0112] (11);

[0113] In formula (11), is the future cumulative return of the coal-fired power plant i from day t, is the discount factor and satisfies 0 ≦ ≦ 1, The smaller it is, the less important the future return is, and the optimization focuses on the recent return; on the contrary, The larger it is, the more important the future return is.

[0114] Further, the action strategy of the coal-fired power plant i is represented by , that is, the probability of a i (t) = a when s(t) = s. The action strategy defines the probability distribution of the possible behaviors of the market subject (that is, the behavior subject) in all states, which determines the action of the market subject when facing different environments.

[0115] In addition, the value function of the state s (s ∈ , is the state space) under the action strategy is recorded as formula (12).

[0116] (12);

[0117] As can be seen from formula (12), the state value function represents the value of all possible actions in this state.

[0118] Similarly, the value function of the action a (a ∈ , is the action space) under the action strategy is recorded as formula (13).

[0119] (13);

[0120] Action value function in formula (13) represents: the value of the market subject performing an action a given the environment state.

[0121] The above is a description of market subject modeling, through market subject modeling, the action strategy State value function under and action value function .

[0122] On the basis of the above "1, description of the power market clearing mechanism" and "2, description of market subject modeling", see Figure 4 , a flowchart of a multi-agent bidding behavior simulation method based on non-independent opponent modeling provided by an embodiment of the present application is shown, which includes the following steps:

[0123] Step S401: load the fitting function in the pre-constructed SOM multi-agent market framework.

[0124] In the process of specifically implementing step S401, the fitting function in the pre-constructed SOM multi-agent market framework is loaded, which is denoted as f.

[0125] It should be noted that in view of the difficulty of multi-agent (Multi-Agent) behavior modeling of complex systems, the present scheme constructs a SOM-based multi-agent market framework, specifically, the present scheme takes into account the characteristics of power market transactions, improves the SOM structure, and designs a SOM-based power market multi-agent inference model.

[0126] In the SOM multi-agent market framework, a fitting function f is given, which is used to calculate the action strategy and state value function In the real power market state, the environment state is completely observable to the market subject, but the bidding strategy and actual behavior of other competing market subjects are unknown. Therefore, the power market multi-agent inference model provided by the present scheme no longer directly incorporates the true actions of the competitors into the model estimation, but uses the market clearing result as a reference. Therefore, the fitting function f takes the actual observed state of the environment, the characteristic attributes of the market subject (such as plant auxiliary power rate, installed capacity, etc.), and the historical clearing price of the market as input variables. Specifically, each market subject i of the power market has the content shown in formula (14).

[0127] (14);

[0128] In formula (14), is the parameter of the fitting function f of the market agent i (or coal-fired power plant i), specifically, , which is the preset parameter shared by the prediction mode and the action mode of the fitting function mentioned later; is the characteristic attribute of the market agent i, is the potential market clearing result (i.e., the estimate of the market clearing result).

[0129] In some specific embodiments, the fitting function f is obtained by reinforcement learning training using an Actor-Critic model, which is composed of a Critic network and an Actor network.

[0130] Under the SOM multi-agent market framework, the fitting function f has two different modes, "prediction mode" and "action mode", the prediction mode is denoted as , and the action mode is denoted as .

[0131] Step S402: For each market agent of the electricity market, when the market agent makes each round of decision, the prediction mode of the fitting function is enabled to infer the action probability distribution of other market agents and estimate the potential market clearing result.

[0132] In the process of implementing step S402, for each market agent (i.e., market agent i) of the electricity market, when the market agent makes each round of decision, the prediction mode of the fitting function is enabled to infer the action probability distribution of other market agents and estimate the potential market clearing result. .

[0133] Obtain the real historical market clearing result (denoted as ), market state (s(t)), and characteristic attributes of other market agents.

[0134] It should be noted that the characteristic attributes of other market agents include but are not limited to: generation self-use rate (see the definition of above), power supply coal consumption coefficient (see the definition of above), electricity coal transaction price factor (see the definition of above), and other fixed costs (see the definition of above).

[0135] Input the real historical market clearing result, market state, and characteristic attributes of other market agents into the fitting function with the prediction mode enabled to infer the bidding strategy of other market agents (Q). .

[0136] Use the preset electricity market clearing mechanism and the bidding strategy of other market agents to estimate the potential market clearing result.

[0137] That is, for each market agent, when the market agent makes each round of decision, the inference mode of the fitting function is enabled; the input variables of the inference mode are: the real historical market clearing result, the market state, and the characteristic attributes of other market agents.

[0138] The bidding and offering strategy of other market agents is inferred by using the fitting function with the inference mode, and the potential market clearing result is estimated by using the power market clearing mechanism mentioned in "1. Description of the power market clearing mechanism" and the bidding and offering strategy of other market agents. .

[0139] Step S403: The action mode of the fitting function is enabled to determine the bidding and offering of the market agent according to the potential market clearing result, and the inference mode and the action mode of the fitting function share a set of preset parameters.

[0140] In the process of specifically implementing step S404, for each market agent (market agent i), when the market agent makes each round of decision, the potential market clearing result is obtained by using the inference mode of the fitting function Then, the action mode of the fitting function is enabled ;

[0141] The market state (s(t)) and the characteristic attributes of the market agent (market agent i) are obtained, and it should be noted that the characteristic attributes of the market agent i include but are not limited to: power generation self-use rate (a ), power supply coal consumption coefficient (b ), electricity coal transaction price factor (c ), and other fixed costs (d ).

[0142] The potential market clearing result , the market state, and the characteristic attributes of the market agent i are input into the fitting function with the action mode to infer the bidding and offering of the market agent i, which includes: the bidding and offering strategy (x ) and the value of the bidding and offering strategy (v ).

[0143] That is, the potential market clearing result is obtained by using the inference mode of the fitting function Then, the action mode of the fitting function is enabled; the input variables of the action mode are: the potential market clearing result , the market state, and the characteristic attributes of the market agent i.

[0144] The bidding and offering of the market agent is determined by using the action mode of the fitting function.

[0145] It should be noted that the prediction mode and the action mode of the fitting function share a set of preset parameters (i.e. ).

[0146] Step S404: using the preset power market clearing mechanism and the bid and offer of all market participants, calculate the real market clearing result.

[0147] In the process of implementing step S404, in each round of decision-making, after obtaining the bid and offer of each market participant through the above steps S402 and S403 (i.e. all market participants have completed decision-making), using the power market clearing mechanism mentioned in "1. Description of the power market clearing mechanism" and the bid and offer of all market participants, calculate the real market clearing result in this round of decision-making.

[0148] In some embodiments, after calculating the real market clearing result in this round of decision-making, update the preset parameters shared by the prediction mode and the action mode of the fitting function according to the real market clearing result, i.e. use the "real market clearing result" to update the parameters .

[0149] In the embodiments of the present application, the possible bids of multiple opponents are considered in the strategy of the market participant, and the self-driven bottom-up simulation of the power market transaction is driven by behavior, so as to obtain a more competitive bidding strategy.

[0150] In order to better understand how to use the prediction mode and the action mode of the fitting function, the SOM-based decision inference and decision framework shown in Figure 5 is described in detail.

[0151] As shown in the SOM-based decision inference and decision framework shown in Figure 5 , each market participant i (such as a coal-fired power plant i, denoted as Agent i) will use its own strategy to simulate the behavior of other market participants, and then decide its own action. The fitting function f provided by the present scheme has two different modes, "prediction mode" and "action mode".

[0152] The action mode is denoted as (abbr. ), in which the market participant i uses the fitting function f to decide what action to take.

[0153] The prediction mode is denoted as (abbr. ), in the speculation mode, market subject i uses the fitting function f to infer the behavior of other market subjects (i.e. other market bidding subjects) and the possible market clearing price. Obviously, since the fitting function f estimates the action strategy and value function (such as the state value function) based on the perspective and decision of market subject i, whether in the action mode or the speculation mode, and All share a set of preset parameters ; The main difference between the two aforementioned models lies in the differences in the characteristic attributes of market entities in the input variables.

[0154] It should be noted that the action mode The input variables are: the market state s(t) at time t, (i.e., various characteristic attributes of market entity i), (Potential market clearing outcome).

[0155] Among them, the various characteristic attributes of market entity i include but are not limited to: power generation self-use rate ( )、Power supply coal consumption coefficient( ), thermal coal transaction price factor ( ), other fixed costs ( ).

[0156] Speculation Mode The input variables are: the market state s(t) at time t, (i.e. various characteristic attributes of other market entities), (Real historical market clearing results).

[0157] Among them, the various characteristic attributes of other market entities include but are not limited to: power generation self-use rate, power supply coal consumption coefficient, power coal transaction price factor, and other fixed costs.

[0158] It should be noted that Figure 5 middle, for the reason The inferred quotation and volume strategies of other market players, for the reason The calculated quotation and volume strategy of oneself (market entity i), for the reason The calculated value of one's own (market entity i) quotation and volume strategy.

[0159] like Figure 5 As shown, when market subject i makes decisions in each round, it first uses the inference model of the fitting function To infer the probability distribution of actions of other market players and then estimate the potential market clearing results In speculative mode, the fitting function does not generate any parameter updates. Then, the “potential market clearing result This estimate was used as a model for action Finally, when all market players have completed their decisions (after receiving their own bids and quantities), the power market clearing mechanism mentioned in "1. Explanation of the power market clearing mechanism" is used to calculate the actual market clearing result of this round, and then the parameters to update.

[0160] The above is an explanation of how to use the inference mode and action mode of the fitting function.

[0161] For the fitting function mentioned above, the specific content of the fitting function is explained in detail through the following content:

[0162] In a multi-agent market framework based on a SOM, the fitting function f plays a crucial role. It not only needs to infer the likely behavior of competitors and the market's transaction outcomes, but also output the agent's own strategy. This process is obviously nonlinear and complex, so this solution uses a neural network to parameterize the behavior of market agents. Specifically, when an agent participates in a bidding process, the observed market state and clearing target serve as inputs to the neural network. Based on its own strategy, the neural network estimates the likelihood that the actions of other competing agents will jointly shape the market's transaction state, and outputs an estimate of the potential future returns.

[0163] In the above bidding process, market players need to choose the best possible bidding action a based on the perceived market environment state s(t). i (t), thereby maximizing the benefits. This involves the optimization of the subject's strategy and the calculation of the value function, which can be achieved through reinforcement learning.

[0164] To this end, this scheme adopts the Actor-Critic model in the fitting function f under the SOM multi-agent market framework. The Actor-Critic model is an algorithm that integrates strategy iteration and value iteration and is composed of two neural networks.

[0165] like Figure 6 The optimization training framework based on Actor-Critic is shown. In the Actor-Critic model, Critic is a judgment network that generates the state value function of the market subject according to the current environment state. Or the action-value function , to evaluate the quality of the strategy generated by the strategy network. The loss function of the evaluation network adopts the mean square error (TD error) of temporal difference, which is shown in formula (15).

[0166] (15).

[0167] Actor is a strategy network that outputs the action strategy of the market subject according to the current market environment status. , the policy network uses the reward gradient to update the network, and its corresponding loss function is shown in formula (16).

[0168] (16);

[0169] In formula (16), is the temporal difference error, coming from the judgment network; represents the optimal strategy.

[0170] Furthermore, based on the above-mentioned SOM-based multi-agent interaction framework and market subject strategy optimization training framework, the pseudo code for SOM-based multi-agent bidding single-round training is as follows:

[0171] 1. MABM Bidding

[0172] 2. Initialize market clearing targets ;

[0173] 3. for t :=1, episode_lengthdo;

[0174] 4. for i :=1, num_playersdo;

[0175] 5. Update state: s(t)←(D DA,t-1 , D RT,t-1 );

[0176] 6. for k :=1, num_inference_stepsdo;

[0177] 7. Opponent strategy update ← ;

[0178] 8. Update market clearing targets ;

[0179] 9. loss = mean_square_loss( , );

[0180] 10、loss.backward();

[0181] 11、policy update, value function update: , ← ;

[0182] 12、report offer and volume: ~ ;

[0183] 13、generate market clearing results ;

[0184] 14、for i :=1, num_playersdo;

[0185] 15、policy.update( )。

[0186] The above is the relevant description of the fitting function.

[0187] In order to verify the effectiveness of the scheme, the following respectively from "data collection and preprocessing" and "simulation results" two parts, the effectiveness of the scheme is illustrated by example.

[0188] Explanation of data collection and preprocessing:

[0189] The data of coal-fired power plants used comes from the statistical yearbook of a certain power and the compilation of statistical data of a certain power industry about the 2021 installed capacity of more than 6MW coal-fired power plants in "SX" province and "SD" province. After data cleaning and correction, the basic statistical information of coal-fired power plants such as installed capacity, power supply coal consumption and auxiliary power rate is shown in table 1 (basic information of coal-fired power plants). In 2021, there were 114 coal-fired power plants with installed capacity of more than 6MW in SX province, and the scale of coal-fired power plants was relatively uniform. It is worth noting that the larger the installed capacity of coal-fired power plants in SX province, the lower the level of power supply coal consumption. That is, when the price of coal purchase is the same, large-scale coal-fired power plants not only have scale advantage, but also have cost advantage when participating in bidding. There are a total of 393 coal-fired power plants with installed capacity of more than 6MW in SD province, and most of them are small-scale power plants, accounting for 74.30% of the total.

[0190] Table 1:

[0191]

[0192] The power market load demand adopted is calculated from the provincial load demand, provincial renewable energy output load, and tie-line load. Among them, the provincial load demand, provincial renewable energy output load, and tie-line load data are derived from designated channels, and the statistical frequency is 15 minutes. In order to be consistent with the power market transaction frequency, it is converted into hourly statistics, and further calculated according to the following formula (17) to calculate the thermal power load demand of the provincial power market.

[0193] D TH =D Load -O RE -D IL (17);

[0194] In formula (17), D TH is the thermal power load demand of the provincial power market, D Load , O RE and-D IL are the provincial load demand, provincial renewable energy output load, and tie-line load, respectively.

[0195] Figure 7 The load demand of SX province from April 2021 to September 2023 and SD province from December 2021 to September 2023 is shown. As can be seen from the “(a)” and “(b)” parts in FIG. 1, Figure 7 the green part is the renewable energy generation load, the dark blue part is the tie-line load, and the yellow part is the provincial other load demand.

[0196] Figure 7 The “(c)” and “(d)” parts in FIG. 2 show the average daily power market thermal power load demand (red line) of SX province and SD province in the sample period. Among them, the green line represents the actual power market thermal power load demand of each day, and the dotted line is the average daily total load demand. Overall, the daily power market thermal power load demand of SX province is close to the total load demand, and the two curves are very close, and the coal-fired power plant plays a very important role in SX province; in comparison, the daily power market thermal power load demand of SD province has a significant gap with the total load demand, and the inter-provincial dispatching and renewable energy output bear more than one-third of the demand.

[0197] The coal purchasing price adopted in the scheme comes from the CECI index. The CECI index is formed after the real coal transaction data is standardized and the calorific value is converted into the final coal purchasing price index. The sample size of the coal transaction data included in the statistics covers more than 80%.

[0198] Figure 8The electricity coal procurement price index (composite price 5500) from April 2021 to September 2023 is shown, where the dark blue curve represents the CECI index, and the red line marks the time when the carbon emission right trading market starts. The yellow part and the green part are the time when the power market in SX province and SD province starts trial operation, respectively.

[0199] It should be noted that, Figure 7 and Figure 8 The content is a description of the data input into the model.

[0200] On the basis of the above “description of data collection and preprocessing”, the “fitting function based on SOM multi-agent market framework” provided in the present solution is used for simulation, so as to calculate the market clearing result.

[0201] Explanation of simulation results:

[0202] First, the predictable demand proportion in the day-ahead market is analyzed, as shown in Figure 9 The green curve and the blue curve represent the average clearing price of the day-ahead market and the real-time market, respectively, and the shaded part marks the corresponding standard deviation interval. Overall, in the day-ahead market, the higher the demand, the higher the day-ahead clearing price. This is because in the day-ahead market, the demand D DA,t represents the theoretical total demand on day t, even if there is a prediction bias, this demand still accounts for more than half of the actual demand on day t. This means that market agents need to ensure successful clearing in this market with high demand first, rather than raising prices. Therefore, in this case, all market agents will adopt a more conservative strategy to clear as the basic goal, showing that when the demand is high, the clearing price is high; on the contrary, when the demand is low, the clearing price is also low.

[0203] As can be seen from Figure 10 , in the 8 simulation scenarios, the average clearing price of the day-ahead market in SX province (the “(a)” part of Figure 10 ) is 202 yuan / MW to 427 yuan / MW, and the average clearing price of the real-time market is 488 yuan / MW to 522 yuan / MW. In comparison, the average clearing price of the day-ahead market in SD province (the “(b)” part of Figure 11The overall clearing price level is lower, with average prices ranging from 172 to 374 RMB / MW in the day-ahead market and 405 to 231 RMB / MW in the real-time market. In addition to differences in actual electricity consumption demand between the two provinces, this significant price difference is due to the number of electricity market participants in the two provinces. SD Province has a total of 352 coal-fired power plants participating in electricity market transactions, nearly four times the number of market participants in SX Province. Therefore, even though SD Province's market load demand is approximately 1.5 times that of SX Province, due to the greater number of market participants, under perfect competition, the market is more competitive and the market clearing price is lower.

[0204] Figure 10 and Figure 11 This chart compares the relationship between electricity demand and clearing prices in the day-ahead and real-time markets in South and Southern Africa under different scenarios with predictable demand ratios. The green and blue scatter plots represent the specific clearing prices and market demand in the day-ahead and real-time markets, respectively. This chart not only demonstrates a positive correlation between clearing prices and daily demand in both markets under the same scenario, but also further confirms that even when demand is decomposed into two components and the amount of electricity to be traded in both markets is equal, the clearing price in the day-ahead market is lower than that in the real-time market.

[0205] This solution also compares the satisfaction of electricity demand under competitive market conditions under various market structure simulation scenarios. Figure 12 As shown, the green and blue colors represent the proportion of power supply shortages in the day-ahead and real-time markets, respectively. Clearly, under perfectly competitive markets, even if the total installed capacity of market participants far exceeds market demand (nearly double), supply shortages can still occur. Comparing the power shortages in SX Province and SD Province, SX Province experiences a higher proportion of power supply shortages because its total installed capacity to market demand is lower than SD Province's.

[0206] Comparing the day-ahead and real-time markets, the real-time market experiences power shortages at a higher rate, approximately 1.5-2 times higher. There are two potential reasons for this phenomenon: first, the real-time market is designed to meet demand fluctuations caused by factors such as forecast bias, volatility in actual demand, and uncertainty in renewable energy output. These fluctuations are highly temporary and unpredictable; second, the installed capacity of market participants is limited, with a considerable portion already occupied through day-ahead market transactions, leaving little capacity available to meet real-time market transactions.

[0207] Further, the relationship between power supply shortage and market demand under different predictable demand ratio scenarios is compared, Figure 13 and Figure 14 The distribution of power demand and power supply shortage in the day-ahead market and the real-time market in SX province and SD province is shown, where the green box plot represents the day-ahead market and the blue box plot represents the real-time market. In the day-ahead market, the smaller the deviation of the forecast of power load demand, the larger the market demand corresponding to the power shortage phenomenon; conversely, in the real-time market, the market demand corresponding to the power shortage phenomenon decreases as the forecast deviation decreases.

[0208] In addition, the clearing hour ratio and the clearing capacity ratio of different installed capacity coal-fired power plants in SX province and SD province are compared, as shown in Figure 15 In SX province, the clearing hour ratio of the market participants in the day-ahead market is about 55%, and the clearing hour ratio in the real-time market is higher, exceeding 60% (part "(a)" of Figure 15 ). Among them, the medium and large coal-fired power plants have a higher clearing hour ratio and have stronger market competitiveness. From the perspective of clearing capacity (part "(b)" of Figure 15 ), the capacity utilization of the medium and large coal-fired power plants is also slightly higher than that of the small and medium-sized power plants. Similarly, in SD province, the large coal-fired power plants are more competitive in the day-ahead market and the real-time market, and their clearing hour ratio and clearing capacity ratio are higher than those of the other three types of installed capacity. However, considering that the number of coal-fired power plants in SD province is large, it is difficult for large coal-fired power plants to form a very significant advantage under competitive market conditions.

[0209] Figure 16 The clearing hour ratio and the clearing capacity ratio of different coal-fired power plants with different power supply coal consumption in SX province and SD province are compared. Obviously, the lower the cost of the coal-fired power plant, the higher the clearing ratio in the power market, and this trend is reflected in both the day-ahead market and the real-time market. As shown in part "(a)" of Figure 16 , the clearing hour ratio of coal-fired power plants with power supply coal consumption less than 100 g / kWh in SX province in the day-ahead market exceeds 70%, and even reaches 88.79% in the real-time market, which is much higher than the 44% clearing hour ratio of coal-fired power plants with power supply coal consumption of 300-400 g / kWh. In terms of clearing capacity ratio, coal-fired power plants with low power supply coal consumption also have a huge advantage, and the capacity utilization of coal-fired power plants with power supply coal consumption less than 100 g / kWh in SX province is about twice that of high coal consumption power plants (300-400 g / kWh) in the day-ahead market and the real-time market. In SD province, the clearing advantage of coal-fired power plants with power supply coal consumption less than 100 g / kWh is more obvious, and their clearing hour ratio exceeds 90% in the day-ahead market and the real-time market (part "(a)" of Figure 16The clearing capacity ratio is also close to 60% (see the "(c)" part of the table) Figure 16 The simulation results mean that, in order to occupy a larger market share, coal-fired power plants need to optimize the power supply coal consumption to reduce production costs, so that the quotation in the electricity market has more comparative advantages.

[0210] The above is the relevant description of the simulation results.

[0211] In summary, the present scheme builds a non-independent decision-making mechanism between agents based on the ABM framework, and proposes a set of multi-agent bidding behavior simulation algorithms based on SOM. This algorithm not only retains the description of agent heterogeneity and interaction with the environment in ABM, but also considers the possible bids of opponents in the strategy of the agent, and drives the simulation of electricity market transactions from bottom to top by behavior. The model uses a deep neural network to fit the inference function, and uses the Actor-Critic reinforcement learning algorithm for strategy iteration and value iteration to achieve decision optimization of each market agent, so that the market agent has better information learning ability. Based on this simulation algorithm, the effectiveness of the model is verified through the market real data of SD province and SX province from 2021 to 2023. The experimental results show the clearing price of the day-ahead and real-time market, the supply security of the competitive electricity market, the relationship between the market price and demand of the time-of-use market, and the market participation of heterogeneous agents. These results will provide strong support for related decision-making of the electricity market.

[0212] Corresponding to the above-mentioned embodiment of the present application provides a kind of multi-agent bidding behavior simulation method based on non-independent opponent modeling, see Figure 17 The embodiment of the present application also provides a structural diagram of multi-agent bidding behavior simulation device based on non-independent opponent modeling, which includes loading unit 100, first enabling unit 200, second enabling unit 300 and computing unit 400.

[0213] The loading unit 100 is used to load the fitting function in the pre-constructed SOM-based multi-agent market framework.

[0214] In some specific embodiments, the fitting function is obtained by reinforcement learning training using an Actor-Critic model, which is composed of a critic network and a policy network.

[0215] The first enabling unit 200 is used to enable the inference mode of the fitting function to infer the action probability distribution of other market agents and estimate the potential market clearing result when each market agent makes a decision in each round of the electricity market.

[0216] The second enabling unit 300 is configured to enable the action mode of the fitting function to determine the bid quantity of the market subject according to the potential market clearing result, and the speculation mode and the action mode of the fitting function share a set of preset parameters.

[0217] The calculation unit 400 is configured to calculate the real market clearing result by using the preset power market clearing mechanism and the bid quantity of all the market subjects.

[0218] In the embodiment of the present application, the possible bids of multiple opponents are considered in the strategy of the market subject, and the power market transaction is simulated from bottom to top in a behavior-driven manner, so that a more competitive bid strategy is obtained.

[0219] Preferably, in combination with Figure 17 As shown, the first enabling unit 200 includes an enabling module, an acquisition module, an input module and an estimation module, and the execution principles of the respective modules are as follows:

[0220] The enabling module is configured to enable the speculation mode of the fitting function for each market subject of the power market when the market subject makes each round of decision.

[0221] The acquisition module is configured to acquire the real historical market clearing result, the market state and the characteristic attributes of other market subjects.

[0222] The input module is configured to input the real historical market clearing result, the market state and the characteristic attributes of other market subjects into the fitting function enabled with the speculation mode to infer the bid strategy of other market subjects.

[0223] The estimation module is configured to estimate the potential market clearing result by using the preset power market clearing mechanism and the bid strategy of other market subjects.

[0224] Preferably, in combination with Figure 17 As shown, the second enabling unit 300 includes an enabling module, an acquisition module and an input module, and the execution principles of the respective modules are as follows:

[0225] The enabling module is configured to enable the action mode of the fitting function.

[0226] The acquisition module is configured to acquire the market state and the characteristic attributes of the market subject.

[0227] The input module is configured to input the potential market clearing result, the market state and the characteristic attributes of the market subject into the fitting function enabled with the action mode to infer the bid of the market subject, and the bid of the market subject includes the bid strategy and the value of the bid strategy.

[0228] Preferably, in combination with Figure 17According to the content shown, the multi-agent bidding behavior simulation device further comprises:

[0229] An updating unit is configured to update preset parameters shared by the speculation mode and the action mode of the fitting function according to the real market clearing result.

[0230] In summary, the embodiment of the present application provides a multi-agent bidding behavior simulation method and device based on non-independent opponent modeling, which considers possible bidding of multiple opponents in the strategy of market agents, and drives the simulation of power market transactions from bottom to top by behavior, so as to obtain a more competitive bidding strategy.

[0231] Each of the embodiments in the specification is described in a progressive manner, and the same and similar parts of each embodiment can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, it is described more simply, and the relevant parts can be referred to the part of the method embodiment. The above-described system and system embodiment are only illustrative, and the units described as separate components can be or can not be physically separated, and the components shown as units can be or can not be physical units, that is, they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0232] The skilled person can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in the above description. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical scheme. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0233] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-agent bidding behavior simulation method based on non-independent opponent modeling, characterized in that: The method comprises: Load the pre-built fitting function based on the SOM multi-agent market framework; For each market player in the electricity market, when the market player makes each round of decision-making, the inference model of the fitting function is activated to infer the action probability distribution of other market players and estimate the potential market clearing results; activating the action mode of the fitting function to determine the quote quantity of the market entity based on the potential market clearing result, wherein the speculation mode and the action mode of the fitting function share a set of preset parameters; The actual market clearing results are calculated using the preset electricity market clearing mechanism and the quoted quantities and quotations of all the market players.

2. The method according to claim 1, characterized in that For each market player in the electricity market, when the market player makes each round of decision-making, the inference model of the fitting function is activated to infer the action probability distribution of other market players and estimate the potential market clearing results, including: For each market entity in the electricity market, when the market entity makes each round of decision-making, the inference mode of the fitting function is enabled; Obtain real historical market clearing results, market status, and characteristic attributes of other market entities; Inputting the real historical market clearing results, the market state, and the characteristic attributes of the other market players into the fitting function with the inference mode enabled to infer the quotation and volume strategies of the other market players; The potential market clearing results are estimated by using the preset electricity market clearing mechanism and the bidding and quantity reporting strategies of other market players.

3. The method according to claim 1, characterized in that Activating the action mode of the fitting function to determine the quote quantity of the market entity based on the potential market clearing result includes: enabling an action mode of the fitting function; Obtaining market status and characteristic attributes of the market entities; The potential market clearing result, the market state, and the characteristic attributes of the market players are input into the fitting function with the action mode enabled to infer the quotation and volume of the market players. The quotation and volume of the market players include: quotation and volume strategy and the value of the quotation and volume strategy.

4. The method according to any one of claims 1 to 3, characterized in that The fitting function is obtained by reinforcement learning training using an Actor-Critic model, where the Actor-Critic model consists of a judgment network and a strategy network.

5. The method according to any one of claims 1 to 3, characterized in that: After calculating the true market clearing results, it also includes: According to the actual market clearing result, the preset parameters shared by the speculation mode and the action mode of the fitting function are updated.

6. A multi-agent bidding behavior simulation device based on non-independent opponent modeling, characterized in that: The device comprises: Loading unit, used to load the pre-built fitting function in the SOM multi-agent market framework; The first activation unit is configured to activate the inference model of the fitting function for each market entity in the electricity market, when the market entity makes each round of decision-making, to infer the action probability distribution of other market entities and estimate potential market clearing results; A second activation unit is configured to activate an action mode of the fitting function to determine the quote quantity of the market entity according to the potential market clearing result, wherein the inference mode and the action mode of the fitting function share a set of preset parameters; The calculation unit is used to calculate the actual market clearing result by using the preset electricity market clearing mechanism and the quantity and quotation of all the market entities.

7. The device according to claim 6, characterized in that The first enabling unit includes: An enabling module, configured to enable the inference mode of the fitting function for each market entity in the electricity market when the market entity makes each round of decision-making; The acquisition module is used to obtain the real historical market clearing results, market status, and characteristic attributes of other market entities; An input module, configured to input the real historical market clearing results, the market state, and the characteristic attributes of the other market players into the fitting function with the inference mode enabled, so as to infer the quotation and volume strategies of the other market players; The estimation module is used to estimate potential market clearing results by utilizing the preset electricity market clearing mechanism and the bidding and quantity reporting strategies of the other market players.

8. The device according to claim 6, characterized in that The second enabling unit includes: An enabling module, used for enabling the action mode of the fitting function; An acquisition module, used to acquire the market status and characteristic attributes of the market entities; An input module is used to input the potential market clearing results, the market status, and the characteristic attributes of the market entities into the fitting function with the action mode enabled, so as to infer the quotation and volume of the market entities, wherein the quotation and volume of the market entities include: quotation and volume strategy and the value of the quotation and volume strategy.

9. The device according to any one of claims 6 to 8, characterized in that The fitting function is obtained by reinforcement learning training using an Actor-Critic model, where the Actor-Critic model consists of a judgment network and a strategy network.

10. The device according to any one of claims 6 to 8, characterized in that: The device further comprises: An updating unit is used to update the preset parameters shared by the speculation mode and the action mode of the fitting function according to the actual market clearing result.

Images