Medium and short term electric power spot price prediction method based on multi-agent simulation

Abstract

The invention relates to the technical field of electric power spot market price prediction, in particular to a medium-short-term electric power spot market price prediction method based on multi-agent simulation, which comprises the following steps: S1, data acquisition: acquiring historical markets, unit parameters and prediction day data; s2, main body optimization: performing dimension reduction by using a Gaussian mixture model, and determining the number of main bodies according to AIC / BIC; s3, environment simulation: according to a marginal price mechanism, determining an electricity price through a supply-demand relationship; s4, a reward function: fusing price, contract deviation and income, and dynamic weight to realize optimization; s5, strategy model establishment: constructing a power generator bidding model based on MADDPG; s6, model training: training and optimizing a power generator bidding strategy model; and S7, prediction evaluation: evaluating a model effect to realize electricity price prediction. According to the method, medium-short-term high-precision prediction of the electric power spot price is realized, and the adaptability and prediction reliability of the model in a complex market environment are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electricity spot market price forecasting, and in particular to a medium- and short-term electricity spot price forecasting method based on multi-agent simulation. Background Art

[0002] The electricity spot market is an important link in promoting the market-oriented reform of the electricity industry. Its price fluctuations directly affect the profit distribution and risk control of market players. Accurate electricity price forecasts can not only provide a scientific basis for market players such as power generators and power sales companies to participate in the game, but also assist them in making reasonable investment and operation decisions, thereby ensuring the stable operation and orderly development of the electricity market. However, the electricity spot price is affected by various internal and external factors such as supply and demand, renewable energy output, meteorological factors, policy regulation and market mechanisms, and shows strong nonlinearity, high volatility and uncertainty, which brings great challenges to electricity price forecasting.

[0003] Existing medium- and short-term electricity price forecasting methods mostly use regression analysis, time series models, machine learning, or neural networks. Their core is to explore the key factors that affect electricity consumption behavior and improve accuracy by building static prediction models. However, such methods are usually highly dependent on feature engineering and have difficulty automatically adapting to dynamic changes in the electricity market. At the same time, as the forecast period lengthens, the forecast error tends to gradually accumulate, resulting in a decrease in forecast accuracy. More importantly, existing methods lack the ability to model market game behavior mechanisms and find it difficult to characterize the dynamic response behavior of entities such as power generators under market rules, limiting the model's generalization ability and practicality in real market scenarios. Summary of the Invention

[0004] The present invention provides a medium- and short-term electricity spot price forecasting method based on multi-agent simulation, which supports market players in formulating scientific and reasonable strategies and reducing their profit risks by providing high-precision electricity price forecast results on medium- and short-term time scales.

[0005] A method for predicting short- to medium-term electricity spot prices based on multi-agent simulation includes the following steps:

[0006] S1, data collection: obtaining historical market information data, unit parameter data and forecast day market information data;

[0007] S2, market entity optimization: Gaussian mixture model (GMM) is used to reduce the dimension of market entities, and the number of entities is selected by calculating Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC);

[0008] S3, build a spot market simulation environment: according to the marginal price mechanism, the price of electricity transactions is determined by market supply and demand;

[0009] S4, reward function system setup: A three-dimensional reward function system integrating price deviation, contract deviation, and market returns, combined with a dynamic weight adjustment mechanism, achieves multi-objective optimization;

[0010] S5, building a unit bidding strategy model: Based on the multi-agent deep deterministic policy gradient (MADDPG) reinforcement learning algorithm, build a power producer bidding strategy model;

[0011] S6, model training: training the bidding strategy model of the power generator and realizing the autonomous optimization process of the bidding strategy model of the power generator;

[0012] S7, Model Evaluation and Electricity Price Forecasting: Evaluate the effectiveness of the power generator bidding strategy model based on empirical values ​​and realize the prediction of electricity spot prices in the medium and short time scales.

[0013] Optionally, the collected data in S1 includes:

[0014] S11, historical market information data: daily historical market information includes 96 data, forming a historical data set [date, time, demand, clearing price] n×96 , where date represents date, time represents time, demand represents electricity demand, clearingprice represents market clearing price, and n represents the number of days;

[0015] S12, unit parameter data: The parameter data set of each unit is expressed as:

[0016] [unit ID,Capacity,Utilization Hour,

[0017] Coal Consumption, Power Consumption Rate];

[0018] Among them, unitID represents the unit number, Capacity represents the rated capacity of the unit, UtilizationHour represents the average annual operating hours of the power plant, CoalConsumption represents the amount of coal consumed per kilowatt-hour of electricity generated, and PowerConsumptionRate represents the percentage of power consumption per unit time to power generation of the power plant.

[0019] S13, forecast day market information data: obtain the forecast day market information data p n×24 , including 24 data per day.

[0020] Optionally, the market entity optimization in S2 includes:

[0021] S21, probability density function generation: using Gaussian mixture model (GMM), let each data point x in the data set i Each is generated by one of the K Gaussian distributions, and the data point x is obtained i The probability density function of is expressed as:

[0022]

[0023] Where K is the number of Gaussian components, π k is the mixing coefficient of the kth Gaussian distribution;

[0024] S22, parameter estimation: Expectation maximization (EM) algorithm is used to solve, including:

[0025] Expectation step (E step): According to the parameters of the currently estimated Gaussian mixture model (mean μ k , covariance matrix Σ k and the mixing coefficient π k ), calculate for each data point x i The posterior probability Υ belongs to each Gaussian distribution k ik , expressed as:

[0026]

[0027] Maximization step (M step): using the calculated posterior probability Υ ik , update the parameters of each Gaussian distribution, expressed as:

[0028]

[0029] Repeated iteration: Continuously iterate and update the Gaussian mixture model parameters θ until the value of the log-likelihood function lnL(μ,Σ,π) converges or reaches the predetermined number of iterations. The likelihood function is expressed as:

[0030]

[0031] S23, Clustering effect evaluation: The fitting effect of the Gaussian mixture model was evaluated by calculating the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC).

[0032] Optionally, the step of building a spot market simulation environment in S3 includes:

[0033] S31, clearing rule setting: All generators are divided into five segments to declare their supply. The electricity declared in each segment is evenly distributed. The supply information of different time periods is sorted according to the price, with one day as the unit and one hour as a period. The quotation of each generator i in the quotation period s is p i,s , the power generation is g i,s , for each period, according to pi,s Trade based on supply information and demand quantities sorted from low to high, with price taking precedence over price.

[0034] S32, market clearing: If the total transaction capacity ≥ total demand D, the market clearing state of supply and demand balance is reached, and the bid of the last transaction unit is the clearing price. If the total declared capacity is less than the total demand, all supplies are traded, and the clearing price is the highest declared price. If there are multiple identical bids for the last transaction unit, the electricity of these units needs to be redistributed proportionally, and the unsuccessful units are added to the result list to obtain the final transaction result.

[0035] Optionally, the reward function system setting in S4 includes:

[0036] S41, price deviation: The price deviation R of the clearing electricity price at each time point is measured by calculating the square of the relative error between the marginal price and the actual electricity price. p , price deviation reward value R price_t is the deviation of the clearing electricity price at all time nodes, expressed as:

[0037]

[0038] R price_t =-β∑R p ;

[0039] Among them, P clear is the market clearing electricity price at each time of a day, P actual is the actual electricity price at each time of the day, and β is the weight coefficient;

[0040] S42, Contract Deviation: Calculate the annual cumulative power deviation R by calculating the square of the relative error between the cumulative load rate and the target cumulative load rate. gen,t , which is used to evaluate the impact of the agent's quotation strategy on the trading volume and set the time-sensitive weight coefficient γ to guide the agent's quotation strategy to approach the predetermined target state, expressed as:

[0041]

[0042] Among them, G cum,t Refers to the cumulative load rate up to day t, G target,t is the average load factor;

[0043] γ=γ initial +0.001t;

[0044] R gen,t =-γR e ;

[0045] Among them, γ initialRefers to the initial weight coefficient, t is the current time step;

[0046] S43, Market Revenue: Based on the actual power generation of the unit, the market clearing price and its power generation cost (coal consumption cost), calculate the total revenue and total cost of the market entity in one day, and subtract them to get the net revenue value R income , used to evaluate the economic performance of the agent's daily quotation strategy, expressed as:

[0047]

[0048] Among them, Q i,t is the power generation of market entity i at time t, P clear,t is the node marginal price (market clearing price) at time t, Q i,t is the coal consumption per unit power generation of market entity i, and coal_price is the coal price.

[0049] Optionally, the building of the unit bidding strategy model in S5 includes:

[0050] S51, Agent and Environment: Model the bidding decision-making behavior of the power generation side as a Markov game process, define each unit as an agent with the ability to interact with the environment, regard the spot market as the external environment of the agent, and use the MADDPG algorithm to simulate the dynamic behavior evolution process of the power producer;

[0051] S52, Observation state (S): The observation state includes the transaction volume G of unit i i 、Coal consumption C i , the next day's load demand D next and the market clearing price P clear ;

[0052] S53, Action Space (A): At each time t, the agent outputs an action value a through the policy network based on its current local state i , and the action vector a i Divided into two parts, including the first half a i(1) and the second half a i(2) , with probability p(t) for a i(1) Add Gaussian noise n1~N(0,σ 2 ), the action after processing is a' i(1) =a i(1) +n1, with probability p(t) for a i(2) Add Gaussian noise n2~N(0,σ 2 ), the action after processing is a' i(2) =a i(2) +n2, expressed as:

[0053] a i=[(g i,1 ,…,g i,5 ),(p i,1 ,…,p i,5 )];

[0054]

[0055] a' i =[a' i(1) ,a' i(2) ];

[0056] S54, reward value (R): reward value r i Including price deviation R price_t , Contract Deviation R gen,t and net income value R income ;

[0057] S55, network structure: The network structure includes a policy network (Actor) and a value network (Critic). The policy network (Actor) is a multi-layer perceptron (MLP) structure, in which normalization is performed through the Softmax function to characterize the probability distribution of the unit's declared quantity. Through regularization processing, the bidding behavior of the intelligent agent from low to high is simulated. The value network (Critic) adopts a multi-layer perceptron structure.

[0058] Optionally, the model training in S6 includes:

[0059] S61, model initialization: Initialize the policy network A for each agent i i and value network C i , target strategy network TA i and target value network TC i , initialize the experience replay pool D and action noise p0, and set the total number of training rounds M and the time length of the data T;

[0060] S62, autonomous optimization: Each agent gradually finds the optimal strategy through self-learning and iterative optimization in the continuous action space to maximize the cumulative reward;

[0061] S63, training data storage and model dictionary archiving: After each round of iteration, all interaction data generated in the current round are serialized and saved to disk. At the same time, after every 5 training rounds, the parameter dictionary of the current policy network and value network is saved as a checkpoint file.

[0062] Optionally, the autonomous optimization in S62 includes:

[0063] S621, Action Selection and Execution: For each training round (from 1 to M), perform initialization of the environment and time step iteration, and for each time step T (from 1 to the maximum round length), perform action selection, environment interaction, and experience storage for each agent;

[0064] S622, target value calculation: If D>minimum capacity, each agent will randomly extract B samples from D and pass the target value network TC i Calculate the target value y i , expressed as:

[0065] y i =r i +μTC i (S' combined ,A)(1-d i );

[0066] S' combined =[G i ,C i ,D next ,P clear ];

[0067] A=[a'1,a'2,…,a' n ];

[0068] Among them, r i is the immediate reward of the i-th agent at the current time step, μ is the discount factor, S' combined Represents the state set of agent i, including the transaction results G of each agent i i and coal consumption C i and the next day's market demand D next and real-time market price P clear , A is the final action set of all subjects, d i is the termination signal of the i-th agent;

[0069] S623, network parameter update: by minimizing the loss function L C Optimize the value network to maximize the output of the value network (the Q value of the action in the current state) to optimize the policy network. At the same time, add regularization and action penalties to stabilize the training, and use the optimizer to adjust the policy network parameters θ A and the value network parameter θ C Perform back propagation and update, expressed as:

[0070]

[0071] Among them, λ is the weight of the square penalty of the action, α is the weight of the regularization term, and L regularization is the regularization loss of the policy network;

[0072]

[0073] Among them, α A is the learning rate of the policy network, α C is the learning rate of the value network, represents the gradient of the policy network loss function with respect to the policy network parameters, Represents the gradient of the value network loss function with respect to the value network parameters;

[0074] S624, target network soft update: soft update the target network parameters for each agent, expressed as:

[0075] θ TA ←θ TA ×(1-τ)+θ A ×τ;

[0076] θ TC ←θ TC ×(1-τ)+θ C ×τ;

[0077] Among them, θ TA and θ TC are the target policy network parameters and the target value network parameters respectively, and τ is the update coefficient.

[0078] Optionally, the model evaluation and electricity price prediction in S7 include:

[0079] S71, Model Evaluation: Based on the data from the training process, evaluate the overall convergence performance of the power generator bidding strategy model in the market environment;

[0080] S72, predicting electricity prices: Based on the loaded and trained bidding strategy model of the power producer, the input data of the time period to be predicted is used as the input stream to generate a dynamic prediction sequence of the spot electricity price.

[0081] Beneficial effects of the present invention:

[0082] The present invention adopts a Gaussian mixture model to reduce the dimensionality of market entities and combines the Akaike Information Criterion and the Bayesian Information Criterion for clustering optimization, effectively alleviating the dimensionality disaster problem caused by high-dimensional data in the power market, improving data processing efficiency and modeling accuracy. At the same time, it combines the marginal clearing mechanism to construct a spot market simulation environment, which can restore the decision-making logic of power producers under relatively realistic supply and demand game conditions.

[0083] The present invention constructs a bidding strategy model for power producers based on the MADDPG algorithm and sets a multi-dimensional reward function system that integrates price deviation, contract deviation and market returns. This realizes a multi-agent autonomous learning architecture of centralized training and distributed execution, so that the model still has good convergence and generalization capabilities under limited information conditions. Ultimately, it can output a dynamic electricity price forecast sequence suitable for medium- and short-term scenarios, significantly improving the timeliness and accuracy of the forecast, and meeting the intelligent needs of actual power market transactions and scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0085] Figure 1 Schematic diagram of the prediction method flow in an embodiment of the present invention;

[0086] Figure 2 Schematic diagram of the MADDPG algorithm flow in an embodiment of the present invention;

[0087] Figure 3 Schematic diagram of AIC and BIC evaluation indicators for Gaussian mixture clustering according to an embodiment of the present invention;

[0088] Figure 4 This is a schematic diagram of the changes in the accumulated experience value of each agent on a certain day in an embodiment of the present invention;

[0089] Figure 5 Schematic diagram for comparing prediction effects of embodiments of the present invention. DETAILED DESCRIPTION

[0090] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Those skilled in the art may also adopt other alternatives to implement some known technologies; and the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0091] like Figure 1-Figure 5 As shown in FIG, a method for predicting short- to medium-term electricity spot prices based on multi-agent simulation includes the following steps:

[0092] S1, data collection: obtaining historical market information data, unit parameter data and forecast day market information data;

[0093] S2, market entity optimization: Gaussian mixture clustering algorithm (GMM) is used to reduce the feature dimension of market entities, and the number of entities is selected by calculating Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC);

[0094] S3, build a spot market simulation environment: according to the marginal price mechanism, the price of electricity transactions is determined by market supply and demand;

[0095] S4, reward function system setup: A three-dimensional reward function system integrating price deviation, contract deviation, and market returns, combined with a dynamic weight adjustment mechanism, achieves multi-objective optimization;

[0096] S5, building a unit bidding strategy model: Based on the multi-agent deep deterministic policy gradient (MADDPG) reinforcement learning algorithm, build a power producer bidding strategy model;

[0097] S6, Model Training: Training the multi-agent simulation model using reinforcement learning algorithms and implementing the model’s own autonomous optimization process;

[0098] S7, Model Evaluation and Electricity Price Forecasting: Evaluate the effectiveness of the model based on empirical values ​​and achieve electricity spot price forecasts on medium and short-term time scales.

[0099] The collected data in S1 include:

[0100] S11, historical market information data: daily historical market information includes 96 data, forming a historical data set [date, time, demand, clearing price] n×96 Wherein, date represents date, time represents time, demand represents electricity demand, clearingprice represents market clearing price, and n represents the number of days;

[0101] S12, unit parameter data: Each unit's information includes the following dimensions, forming the unit technical parameter dataset: [unit ID, Capacity, Utilization Hour, Coal Consumption, Power Consumption Rate]. Unit ID represents the unit ID, Capacity represents the rated capacity of the unit, Utilization Hour represents the average annual operating hours of the power plant, Coal Consumption represents the amount of coal consumed per kilowatt-hour of electricity generated, and Power Consumption Rate represents the percentage of electricity generated per unit time.

[0102] S13, forecast day market information data: obtain the forecast day market information data p n×24, including 24 data per day.

[0103] Market entity optimization in S2 includes:

[0104] S21, Probability density function generation: The GMM algorithm assumes that each data point x in the data set i are generated by one of the K Gaussian distributions, and the data point x is obtained i The probability density function of is expressed as:

[0105]

[0106] Where K is the number of Gaussian components, π k is the mixing coefficient of the kth Gaussian distribution and satisfies

[0107] S22, parameter estimation: using the expectation maximization (EM) algorithm to solve. Expectation step (E step), according to the current estimated model parameters (mean μ k , covariance matrix Σ k and the mixing coefficient π k ), calculate for each data point x i The responsiveness Υ belonging to each Gaussian distribution k ik , expressed as:

[0108]

[0109] Maximization step (M step), using the calculated posterior probability Υ ik , update the parameters of each Gaussian distribution, the specific update formula is expressed as:

[0110]

[0111] Repeat the EM steps and iteratively update the model parameters θ until the value of the log-likelihood function lnL(μ,Σ,π) converges or the predetermined number of iterations is reached. The likelihood function is expressed as:

[0112]

[0113] S23, Model Evaluation: The Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) were used to evaluate the model's fit. A smaller AIC value indicates a better balance between model fit and model complexity. Similarly, a smaller BIC value indicates a better model.

[0114] The spot market simulation environment in S3 includes:

[0115] S31, clearing rule setting: Assume that all generators declare their supply in five segments, where the electricity declared in each segment is evenly distributed. Take one day as the unit and one hour as a period. Sort the supply information in different time periods by price. The quotation of each generator i in the quotation period s is p i,s , the power generation is g i,s , for each period, according to p i,s Trade based on supply information and demand quantities sorted from low to high, with price taking precedence over price.

[0116] S32, Market Clearing: If the total traded capacity ≥ total demand D, the market clears, achieving supply-demand equilibrium. The bid of the last unit to be traded becomes the clearing price. If the total bid capacity is less than total demand, all supply is traded, and the clearing price is the highest bid. If there are multiple identical bids for the last unit to be traded, the power of these units is redistributed proportionally, and the unsuccessful units are added to the result list to determine the final trade result.

[0117] The reward function system settings in S4 include:

[0118] S41, price deviation: the clearing price deviation R at each time point p , measured by calculating the square of the relative error between the marginal price and the actual electricity price. Price deviation reward value R price_t is the deviation of the clearing electricity price at all time nodes, expressed as:

[0119]

[0120] R price_t =-β∑R p ;

[0121] Among them, P clear is the market clearing price at each time of a day, that is, the marginal price, P actual is the actual electricity price at each time of the day, and β is the weight coefficient. price_t The agents can be encouraged to bid close to the market clearing price, thus reducing the price deviation to approach the true value.

[0122] S42, contract deviation: specifically refers to the annual cumulative power deviation R gen,t , which is used to evaluate the impact of the agent's quotation strategy on trading volume. The present invention calculates the reward by calculating the square of the relative error between the cumulative load rate and the target cumulative load rate, expressed as:

[0123]

[0124] Among them G cum,t Refers to the cumulative load rate up to day t, G target,tis the average load rate. At the same time, in order to reflect the temporal progression of the deviation penalty, a time-sensitive weight coefficient γ is set to guide the agent's bidding strategy to be closer to the predetermined target state. Its calculation formula is:

[0125] γ=γ initial +0.001t;

[0126] R gen,t =-γR e ;

[0127] where γ initial Refers to the initial weight coefficient, and t is the current time step.

[0128] S43, market income: is an economic reward indicator, the reward value R income It is used to evaluate the economic performance of the agent's daily bidding strategy. Based on the actual power generation of the unit, the market clearing price and its power generation cost (coal consumption cost), the total revenue and total cost of the market entity in a day are calculated and subtracted to obtain the net profit value, which is expressed as:

[0129]

[0130] where Q i,t is the power generation of market entity i at time t, P clear,t is the node marginal price (market clearing price) at time t, Q i,t is the coal consumption per unit of power generation for market entity i, and coal_price is the coal price. The market revenue reward value can intuitively reflect the economic efficiency and profitability of the agent's bidding strategy and is a key indicator for measuring the quality of the strategy.

[0131] The bidding strategy model of power generators in S5 includes:

[0132] S51, Agents and Environment: This paper proposes a bidding strategy model for power producers based on multi-agent deep reinforcement learning. The bidding decision-making behavior on the generation side is modeled as a Markov game process. Each unit is defined as an agent capable of interacting with the environment. The spot market is considered the external environment of the agent. The MADDPG algorithm is used to simulate the dynamic behavior evolution of the power producer.

[0133] S52, Observation state (S): refers to the information received by the agent from the environment. The observation state of the present invention consists of four parts, namely, the transaction volume G of unit i i 、Coal consumption C i , the next day's load demand D next and the market clearing price P clear. The transaction volume of the unit refers to the amount of electricity actually traded by the unit in the previous market clearing cycle. This information helps the unit predict its production efficiency and income, helps the intelligent agent understand its performance in the market, and adjusts its quotation strategy accordingly. Coal consumption refers to the amount of coal consumed by the unit in the process of producing electricity. In the embodiment, it can be read directly from the data. This is a key indicator for measuring the operating cost of the unit. The load demand for the next day is the electricity demand for the next day in the data. Understanding this demand can help the unit clarify the market supply and demand relationship, predict market conditions, and thus formulate a more effective quotation strategy. Finally, there is the market clearing electricity price, which is the indicator that the present invention focuses on. This price is a key signal in the electricity market, which helps the intelligent agent understand the market pricing mechanism and adjust its quotation accordingly to maximize profits. These observations will be input into the neural network of each intelligent agent as key information for its interaction with the environment, and become part of the learning process, so that the intelligent agent can have a more comprehensive understanding of the market status and the impact of its own behavior in the process of interacting with the environment.

[0134] S53, Action Space (A): specifically refers to the set of all possible actions that an agent can perform. MADDPG is an extension of the DDPG algorithm in a multi-agent environment, designed to solve the reinforcement learning challenges in multi-agent scenarios, where the algorithm adopts the "centralized training, distributed execution" (DTDE) paradigm for training. DTDE means that all agents share a centralized Critic network. The Critic network can access the information of all agents during training and provide guidance to the Actor network of each agent. However, during execution, the Actor network of each agent takes action completely independently, that is, decentralized execution is achieved. Specifically, at each time t, the agent can directly output the action value a through the policy network based on its current local state. i , expressed as:

[0135] a i =[(g i,1 ,…,g i,5 ),(p i,1 ,…,p i,5 )];

[0136] The action vector a i It is processed in two parts, a i(1) Indicates the first half, a i(2) This is the second half. Time t is equivalent to the current step number, and t0 is the step number from which the noise probability is adjusted. i(1) Add Gaussian noise n1~N(0,σ 2 ), the action after processing is a' i(1) =a i(1) +n1; also with probability p(t) for a i(2)Add Gaussian noise n2~N(0,σ 2 ), the action after processing is a' i(2) =a i(2) +n2, expressed as:

[0137]

[0138] The agent performs a new action a' i Interact with the environment, represented as:

[0139] a' i =[a' i(1) ,a' i(2) ];

[0140] S54, reward value (R): refers to the feedback obtained by the agent from the environment, which is used to evaluate the performance of the agent in performing a certain action. i Set as price deviation R price_t , Contract Deviation R gen,t and market return R income The integration of the three parts. Each time the agent interacts with the environment, it will generate a termination signal d i (If it is 1, it means the terminal state, and 0 means the non-terminal state), the reward r i and the next moment state s' i In order to improve the stability and efficiency of training, MADDPG uses an experience replay mechanism. The state, action, reward, next state and end flag of each interaction of the agent (s i ,a' i ,r i ,s' i ,d i ) will be stored in the experience replay pool D, and a batch of experiences will be randomly extracted from this buffer to update the network during training.

[0141] S55, Network Structure: This paper discloses a unit bidding strategy model based on the MADDPG reinforcement learning algorithm. This algorithm uses an independent actor-critic framework for each agent. The strategy network (actor) determines the agent's action value, namely the power consumption and price reported by the generator, while the value network (critic) evaluates the action and provides corresponding rewards.

[0142] The Actor network is a typical multi-layer perceptron (MLP) architecture, consisting of two fully connected layers that output action probability distributions. The network introduces nonlinear features using the ReLU activation function and applies a scaling factor to regulate the activation value range. To achieve the agent's target action, the output is processed in two parts: the first half is normalized using the Softmax function to ensure that the sum of all elements is 1, and a minimum constraint is imposed to represent the probability distribution of the unit's reported quantity. The second half of the output is regularized to ensure that the output values ​​are unique and arranged in ascending order, thus simulating the agent's bidding behavior from low to high. The Critic network also adopts a multi-layer perceptron structure, consisting of multiple hidden layers and an output layer. Its characteristic is that the number of neurons in the hidden layers decreases with each layer, until the output layer transforms the input state-action pair into a value estimate. This layer-by-layer reduction in neuron configuration enables the Critic network to learn complex function mappings, thereby accurately assessing the value of input data and providing a reliable basis for decision-making in reinforcement learning.

[0143] like Figure 2 As shown, model training in S6 includes:

[0144] S61, model initialization: Initialize the Actor network A for each agent i i and Critic network C i , target Actor network TA i and target critic network TC i Initialize the experience replay pool D and action noise p0, set the total number of training rounds M and the time length of the data T;

[0145] S62, Autonomous Optimization: Each agent gradually finds the optimal strategy through self-learning and iterative optimization in the continuous action space to maximize the long-term cumulative reward. This autonomous optimization process requires no human intervention and is driven entirely by the algorithm itself.

[0146] The autonomous optimization process in the S62 includes:

[0147] S621, Action Selection and Execution: For each training round (from 1 to M), perform initialization of the environment and time step iteration. For each time step T (from 1 to the maximum round length), each agent performs action selection, environment interaction, and experience storage.

[0148] S622, target value calculation: If D>minimum capacity, each agent will randomly extract B samples from D and pass the target network TC i Calculate the target value y i , expressed as:

[0149] yi =r i +μTC i (S' combined ,A)(1-d i );

[0150] S' combined =[G i ,C i ,D next ,P clear ];

[0151] A=[a'1,a'2,…,a' n ];

[0152] Among them, r i is the immediate reward of the i-th agent at the current time step, and μ is the discount factor. combined Represents the state set of agent i, including the transaction results G of each agent i i and coal consumption C i and the next day's market demand D next and real-time market price P clear , A is the final action set of all subjects, d i is the termination signal of the i-th agent.

[0153] S623, network parameter update: by minimizing the loss function L C Optimize the Critic network, expressed as:

[0154]

[0155] The policy network is optimized by maximizing the output of the Critic network (the Q value of the action in the current state), while adding regularization and action penalties to stabilize training, which can be expressed as:

[0156]

[0157] Among them, λ is the weight of the square penalty of the action, α is the weight of the regularization term, and L regularization is the regularization loss of the policy network. Use the optimizer to adjust the Actor network parameters θ A And the critic network parameter θ C Perform back propagation and update, expressed as:

[0158]

[0159] Among them, α A is the learning rate of the Actor network, α C is the learning rate of the Critic network, represents the gradient of the Actor loss function with respect to the Actor network parameters, Represents the gradient of the Critic loss function with respect to the Critic network parameters.

[0160] S624, target network soft update: soft update the target network parameters for each agent, expressed as:

[0161] θ TA ←θ TA ×(1-τ)+θ A ×τ;

[0162] θ TC ←θ TC ×(1-τ)+θ C ×τ;

[0163] Among them, θ TA and θ TC are the target Actor network parameters and the target Critic network parameters, respectively, and τ is the update coefficient;

[0164] S63, training data storage and model dictionary archiving: After each round of iteration, all interaction data generated in the current round are serialized and saved to disk. At the same time, after every 5 training rounds, the parameter dictionary of the current Actor network and Critic network is saved as a checkpoint file to support rapid recovery or strategy comparison after training interruption.

[0165] Model evaluation and electricity price forecasting in S7 include:

[0166] S71, Model Evaluation: Evaluate the overall convergence performance of deep reinforcement learning models in a market environment based on training data. Quantitatively analyze and evaluate the historical reward values ​​and bidding strategies of typical market participants.

[0167] S72, predicting electricity prices: Load the trained deep reinforcement learning model, use the input data of the time period to be predicted as the input stream, and generate a dynamic prediction sequence of spot electricity prices through model strategy deduction.

[0168] In this embodiment, the historical market information of a province's spot market is shown in Table 1. The start and end times of each day are 00:15 and 24:00, respectively, with an interval of 15 minutes, and 96 numbers per day.

[0169] Table 1 Historical market data

[0170]

[0171] Data analysis revealed upper and lower limits for unit bids. Therefore, when setting up the simulation environment, a price cap was set for the electricity spot market, with an upper limit of 1,800 yuan / MWH and a lower limit of -50 yuan / MWH. Although the market clearing frequency is set every 15 minutes, the clearing price remains fixed within an hour. Therefore, to improve the simulation model's performance, the 96 daily data points were converted to 24-point data. This means taking the arithmetic average of the four 15-minute data points per hour as the representative value for that hour.

[0172] The adjusted data format is shown in the following table. The start and end times of each day are 0:00 and 23:00 respectively, with an interval of 1 hour and 24 numbers per day.

[0173] Table 2 Processed historical market data

[0174]

[0175]

[0176] In addition, it also includes the technical parameters of 549 power generation units, as shown in Table 3, covering key indicators such as unit ID, type, installed capacity, power supply coal consumption and utilization hours. In the multi-agent decision-making behavior simulation, directly incorporating all subjects will face three major challenges: large data scale, reduced computing efficiency, and difficulty in model generalization. In order to effectively solve the above problems, the present invention introduces a Gaussian mixture clustering algorithm to perform dimensionality reduction and screening on market entities. The model is evaluated by calculating the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC). The lower the AIC and BIC values, the higher the goodness of fit of the model. At the same time, the complexity of the model is lower, which is the optimal number of clusters. As Figure 3 As shown in the figure, 42 cluster centers were finally selected as representatives of typical market entities, and then a multi-agent simulation model was constructed.

[0177] Table 3 Market participant information

[0178]

[0179] The model uses relevant historical data for 12 months in 2022 as the training environment, sets the clearing frequency to 1 hour, sets the 24 hours of each day as the time interval for a training session, and sets the algorithm training rounds to 200 rounds.

[0180] like Figure 4The figure shows the evolution of each agent's cumulative experience value on a given day during model training. Each subgraph corresponds to a single agent, with the horizontal axis representing the number of training episodes and the vertical axis representing the cumulative reward value. It can be observed that in the early stages of reinforcement learning training, the agents' cumulative reward values ​​often exhibit significant fluctuations, as they actively experiment with different strategies to gain experience. As training progresses, the agents gradually reduce this exploration behavior and increasingly apply their learned strategies, resulting in reduced reward volatility and a more stable trend. After approximately 50 episodes of learning, the agents' exploration behavior further decreases, and they begin to rely more heavily on accumulated bidding experience, leading to a continuous increase in their reward values. This growth trend continues until approximately 120 episodes, at which point the cumulative reward values ​​of most agents show signs of convergence, with further reduced fluctuations. This indicates that the agents' learning process has reached a relatively stable state, and their bidding strategies have become more mature and effective.

[0181] according to Figure 4 As a result, we can observe that after a period of learning, the agents' experience values ​​tend to stabilize. This indicates that most agents have found relatively effective strategies and are able to consistently apply these strategies to gain experience. Therefore, the model training results are relatively ideal, allowing for in-depth predictive analysis.

[0182] The present invention uses the above model to predict the clearing electricity price for the next 31 days, a total of 744 time points, based on historical market clearing prices. Here, the model is compared with the long short-term memory network (LSTM), support vector machine regression (SVR), and random forest (RF) models. The predicted mean absolute error (MAE), mean square error (MSE), and mean absolute percentage error (MAPE) values ​​are shown in Table 6. The lower the MAE, MSE, and MAPE, the better the model's prediction effect. The results show that the proposed method exhibits excellent overall performance across various evaluation indicators, indicating that the deviation between its prediction results and the true value is small and the stability is stronger, demonstrating the advantages of this method in actual prediction tasks.

[0183] In addition, we also specially selected the data of four consecutive days from the 26th to the 29th day of the forecast period and expanded and restored it to the format of 96 time points to match the time resolution of the original data. The comparison effect is as follows: Figure 5 For this scenario, a multi-agent simulation model using the MADDPG algorithm can more sensitively capture peaks, valleys, and flat periods in electricity prices. This not only allows it to better track subtle changes in electricity prices and respond promptly to price fluctuations, but also effectively prevents predictions from lagging or deviating from actual fluctuations.

[0184] Table 6 Comparison of model prediction results

[0185]

[0186] In summary, the application of this patented method significantly improves forecasting performance on short- to medium-term timescales compared to other traditional methods (LSTM, SVR, RF), especially under extreme electricity price conditions. This provides a scientific basis for market participants to make decisions in the electricity spot market, thereby increasing market returns.

[0187] The present invention encompasses any alternatives, modifications, equivalents, and solutions that fall within the spirit and scope of the present invention. To provide a thorough understanding of the present invention, specific details are described in detail below in connection with the preferred embodiments of the present invention, but those skilled in the art will be able to fully understand the present invention without these detailed descriptions. Furthermore, to avoid unnecessary confusion regarding the essence of the present invention, well-known methods, processes, procedures, components, and circuits have not been described in detail.

[0188] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for predicting short- to medium-term electricity spot prices based on multi-agent simulation, characterized in that: The following steps are involved: S1, data collection: obtaining historical market information data, unit parameter data and forecast day market information data; S2, market subject optimization: using Gaussian mixture model to reduce the feature dimension of market subjects, and selecting the number of subjects by calculating Akaike information criterion and Bayesian information criterion; S3, build a spot market simulation environment: according to the marginal price mechanism, the price of electricity transactions is determined by market supply and demand; S4, reward function system setup: A three-dimensional reward function system integrating price deviation, contract deviation, and market returns, combined with a dynamic weight adjustment mechanism, achieves multi-objective optimization; S5, building a unit bidding strategy model: Based on the multi-agent deep deterministic policy gradient reinforcement learning algorithm, build a power producer bidding strategy model; S6, model training: training the bidding strategy model of the power generator and realizing the autonomous optimization process of the bidding strategy model of the power generator; S7, Model Evaluation and Electricity Price Forecasting: Evaluate the effectiveness of the power generator bidding strategy model based on empirical values ​​and realize the prediction of electricity spot prices in the medium and short time scales.

2. A method for predicting short- to medium-term electricity spot prices based on multi-agent simulation according to claim 1, characterized in that: The collected data in S1 includes: S11, historical market information data: daily historical market information includes 96 data, forming a historical data set [date, time, demand, clearing price] n×96 , where date represents date, time represents time, demand represents electricity demand, clearingprice represents market clearing price, and n represents the number of days; S12, unit parameter data: The parameter data set of each unit is expressed as: [unit ID,Capacity,Utilization Hour, Coal Consumption, Power Consumption Rate]; UnitID represents the unit ID, Capacity represents the rated capacity of the unit, Utilization Hour represents the average annual operating hours of the power plant, Coal Consumption represents the amount of coal consumed per kilowatt-hour of electricity generated, and Power Consumption Rate represents the percentage of power consumption per unit time to power generation of the power plant. S13, forecast day market information data: obtain the forecast day market information data p n×24 , including 24 data per day.

3. The method for predicting short- to medium-term electricity spot prices based on multi-agent simulation according to claim 2, characterized in that: The market entity optimization in S2 includes: S21, probability density function generation: Using Gaussian mixture model, let each data point x in the data set i Each is generated by one of the K Gaussian distributions, and the data point x is obtained i The probability density function of is expressed as: Where K is the number of Gaussian components, π k is the mixing coefficient of the kth Gaussian distribution; S22, parameter estimation: using the expectation maximization algorithm to solve, specifically including: Expected step: Calculate for each data point x based on the parameters of the currently estimated Gaussian mixture model i The posterior probability Υ belongs to each Gaussian distribution k ik , expressed as: Maximization step: using the calculated posterior probability Υ ik , update the parameters of each Gaussian distribution, expressed as: Repeated iteration: Continuously iterate and update the Gaussian mixture model parameters θ until the value of the log-likelihood function lnL(μ,Σ,π) converges or reaches the predetermined number of iterations. The likelihood function is expressed as: S23, Clustering effect evaluation: The fitting effect of the Gaussian mixture model was evaluated by calculating the Akaike Information Criterion and the Bayesian Information Criterion.

4. A method for predicting short- to medium-term electricity spot prices based on multi-agent simulation according to claim 3, characterized in that: The spot market simulation environment in S3 includes: S31, clearing rule setting: All generators are divided into five segments to declare their supply. The electricity declared in each segment is evenly distributed. The supply information of different time periods is sorted according to the price, with one day as the unit and one hour as a period. The quotation of each generator i in the quotation period s is p i,s , the power generation is g i,s , for each period, according to p i,s Trade based on supply information and demand quantities sorted from low to high, with price taking precedence over price. S32, market clearing: If the total transaction capacity ≥ total demand D, the market clearing state of supply and demand balance is reached, and the bid of the last transaction unit is the clearing price. If the total declared capacity is less than the total demand, all supplies are traded, and the clearing price is the highest declared price. If there are multiple identical bids for the last transaction unit, the electricity of these units needs to be redistributed proportionally, and the unsuccessful units are added to the result list to obtain the final transaction result.

5. The method for predicting short- to medium-term electricity spot prices based on multi-agent simulation according to claim 4, characterized in that: The reward function system setting in S4 includes: S41, price deviation: The price deviation R of the clearing electricity price at each time point is measured by calculating the square of the relative error between the marginal price and the actual electricity price. p , price deviation reward value R price_t is the deviation of the clearing electricity price at all time nodes, expressed as: R price_t =-β∑R p ; Among them, P clear is the market clearing electricity price at each time of a day, P actual is the actual electricity price at each time of the day, and β is the weight coefficient; S42, Contract Deviation: Calculate the annual cumulative power deviation R by calculating the square of the relative error between the cumulative load rate and the target cumulative load rate. gen,t , which is used to evaluate the impact of the agent's quotation strategy on the trading volume and set the time-sensitive weight coefficient γ to guide the agent's quotation strategy to approach the predetermined target state, expressed as: Among them, G cum,t Refers to the cumulative load rate up to day t, G target,t is the average load factor; c = c initial +0.001t; R gen,t =-γR e ; Among them, γ initial Refers to the initial weight coefficient, t is the current time step; S43, Market Revenue: Based on the actual power generation of the unit, the market clearing price and its power generation cost, calculate the total revenue and total cost of the market entity in one day, and subtract them to obtain the net profit value R income , used to evaluate the economic performance of the agent's daily quotation strategy, expressed as: Among them, Q i,t is the power generation of market entity i at time t, P clear,t is the node marginal price at time t, Q i,t is the coal consumption per unit power generation of market entity i, and coal_price is the coal price.

6. A method for predicting short- to medium-term electricity spot prices based on multi-agent simulation according to claim 5, characterized in that: The construction of the unit bidding strategy model in S5 includes: S51, Agent and Environment: Model the bidding decision-making behavior of the power generation side as a Markov game process, define each unit as an agent with the ability to interact with the environment, regard the spot market as the external environment of the agent, and use the MADDPG algorithm to simulate the dynamic behavior evolution process of the power producer; S52, Observation state: The observation state includes the transaction volume G of unit i i 、Coal consumption C i , the next day's load demand D next and the market clearing price P clear ; S53, action space: At each time t, the agent outputs the action value a through the policy network according to its current local state i , and the action vector a i Divided into two parts, including the first half a i(1) and the second half a i(2) , with probability p(t) for a i(1) Add Gaussian noise n1~N(0,σ 2 ), the action after processing is a' i(1) =a i(1) +n1, with probability p(t) for a i(2) Add Gaussian noise n2~N(0,σ 2 ), the action after processing is a' i(2) =a i(2) +n2, expressed as: a i =[(g i,1 ,…,g i,5 ),(p i,1 ,…,p i,5 )]; a' i =[a' i(1) ,a' i(2) ]; S54, reward value: reward value r i Including price deviation R price_t , Contract Deviation R gen,t and net income value R income ; S55, Network structure: The network structure includes a policy network and a value network. The policy network is a multi-layer perceptron structure, in which the Softmax function is used for normalization to characterize the probability distribution of the unit's declared quantity. Regularization is used to simulate the agent's bidding behavior from low to high. The value network adopts a multi-layer perceptron structure.

7. A method for predicting short- to medium-term electricity spot prices based on multi-agent simulation according to claim 6, characterized in that: The model training in S6 includes: S61, model initialization: Initialize the policy network A for each agent i i and value network C i , target strategy network TA i and target value network TC i , initialize the experience replay pool D and action noise p0, and set the total number of training rounds M and the time length of the data T; S62, autonomous optimization: Each agent gradually finds the optimal strategy through self-learning and iterative optimization in the continuous action space to maximize the cumulative reward; S63, training data storage and model dictionary archiving: After each round of iteration, all interaction data generated in the current round are serialized and saved to disk. At the same time, after every 5 training rounds, the parameter dictionary of the current policy network and value network is saved as a checkpoint file.

8. A method for predicting short- to medium-term electricity spot prices based on multi-agent simulation according to claim 7, characterized in that: The autonomous optimization in S62 includes: S621, Action Selection and Execution: Initialize the environment and time step iteration for each training round, and perform action selection, environment interaction, and experience storage for each agent at each time step T; S622, target value calculation: If D>minimum capacity, each agent will randomly extract B samples from D and pass the target value network TC i Calculate the target value y i , expressed as: y i =r i +μTC i (S' combined ,A)(1-d i ); S' combined =[G i ,C i ,D next ,P clear ]; A=[a'1,a'2,…,a' n ]; Among them, ri is the immediate reward of the i-th agent at the current time step, μ is the discount factor, S' combined Represents the state set of agent i, including the transaction result Gi and coal consumption C of each agent i i and the next day's market demand D next and real-time market price P clear , A is the final action set of all subjects, d i is the termination signal of the i-th agent; S623, network parameter update: by minimizing the loss function L C Optimize the value network to maximize the output of the value network to optimize the policy network, while adding regularization and action penalties to stabilize the training, and use the optimizer to adjust the policy network parameters θ A and the value network parameter θ C Perform back propagation and update, expressed as: Among them, λ is the weight of the square penalty of the action, α is the weight of the regularization term, and L regularization is the regularization loss of the policy network; Among them, α A is the learning rate of the policy network, α C is the learning rate of the value network, represents the gradient of the policy network loss function with respect to the policy network parameters, Represents the gradient of the value network loss function with respect to the value network parameters; S624, target network soft update: soft update the target network parameters for each agent, expressed as: i TA ←θ TA ×(1-τ)+θ A ×t; i TC ←θ TC ×(1-τ)+θ C ×t; Among them, θ TA and θ TC are the target policy network parameters and the target value network parameters respectively, and τ is the update coefficient.

9. The method for predicting short- to medium-term electricity spot prices based on multi-agent simulation according to claim 8, characterized in that: The model evaluation and electricity price prediction in S7 include: S71, Model Evaluation: Based on the data from the training process, evaluate the overall convergence performance of the power generator bidding strategy model in the market environment; S72, predicting electricity prices: Based on the loaded and trained bidding strategy model of the power producer, the input data of the time period to be predicted is used as the input stream to generate a dynamic prediction sequence of the spot electricity price.

Images