A method for short-term and medium-term electricity spot price forecasting based on multi-agent simulation

Abstract

The present application relates to the technical field of electricity spot market price prediction, and particularly relates to a kind of medium and short-term electricity spot market price prediction method based on multi-agent simulation, comprising the following steps: S1, data acquisition: obtain historical market, unit parameter and forecast day data;S2, subject optimization: dimension reduction with Gaussian mixture model, and the number of subjects is determined according to AIC / BIC;S3, simulation environment: according to marginal price mechanism, determine the electricity price through supply and demand relationship;S4, reward function: fusion price, contract deviation and income, dynamic weight realizes optimization;S5, strategy model: construct power supplier bidding model based on MADDPG;S6, model training: train and optimize power supplier bidding strategy model;S7, prediction evaluation: evaluate model effect, realize electricity price prediction.The present application realizes the medium and short-term high-precision prediction of electricity spot market price, significantly improves the adaptability and prediction reliability of the model in complex market environment.

Description

Technical Field

[0001] This invention relates to the field of electricity spot market price forecasting technology, and in particular to a method for forecasting short- to medium-term electricity spot prices based on multi-agent simulation. Background Technology

[0002] The electricity spot market is a crucial link in promoting electricity market reform. Its price fluctuations directly affect the profit distribution and risk control of market participants. Accurate electricity price forecasting can not only provide a scientific basis for power generators, electricity retailers and other market participants to participate in the game, but also assist them in making reasonable investment and operational decisions, thereby ensuring the stable operation and orderly development of the electricity market. However, electricity spot prices are affected by a variety of internal and external factors such as supply and demand, renewable energy output, meteorological factors, policy regulation and market mechanisms, exhibiting strong nonlinearity, high volatility and uncertainty, which poses a significant challenge to electricity price forecasting.

[0003] Existing short- and medium-term electricity price forecasting methods mostly employ regression analysis, time series models, machine learning, or neural networks. Their core lies in identifying key factors influencing electricity consumption behavior and improving accuracy by constructing static forecasting models. However, these methods typically rely heavily on feature engineering and are difficult to automatically adapt to the dynamic changes in the electricity market. Furthermore, as the forecast period lengthens, the forecast error gradually accumulates, leading to a decrease in forecast accuracy. More importantly, existing methods lack the ability to model market game behavior mechanisms and struggle to depict the dynamic response behavior of power generators and other entities under market rules, thus limiting the model's generalization ability and practicality in real market scenarios. Summary of the Invention

[0004] This invention provides a method for predicting short- to medium-term electricity spot prices based on multi-agent simulation. By providing high-precision electricity price prediction results on a short- to medium-term time scale, it supports market participants in formulating scientific and reasonable strategies and reducing their profit risks.

[0005] A method for predicting short- to medium-term electricity spot prices based on multi-agent simulation includes the following steps:

[0006] S1, Data Collection: Obtain historical market information data, unit parameter data, and forecast daily market information data;

[0007] S2, Market Entity Optimization: Gaussian Mixture Model (GMM) is used to reduce the dimensionality of market entities, and the number of entities is selected by calculating the Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC);

[0008] S3, build a spot market simulation environment: determine the price of electricity trading based on the marginal price mechanism and the market supply and demand relationship;

[0009] S4, Reward Function System Setting: A three-dimensional reward function system that integrates price deviation, contract deviation and market return is used to achieve multi-objective optimization by combining a dynamic weight adjustment mechanism;

[0010] S5, Constructing a generator unit bidding strategy model: Based on the reinforcement learning algorithm of Multi-Agent Deep Deterministic Policy Gradient (MADDPG), construct a generator bidding strategy model;

[0011] S6, Model Training: Train the generator bidding strategy model and implement the generator bidding strategy model's own autonomous optimization process.

[0012] S7, Model Evaluation and Electricity Price Forecasting: Evaluate the effectiveness of the power generator bidding strategy model based on empirical values ​​to achieve electricity spot price forecasting in the short to medium term.

[0013] Optionally, the data collected in S1 includes:

[0014] S11, Historical Market Information Data: Daily historical market information includes 96 data points, forming the historical dataset. ,in, date Indicates the date. time Indicates time, demand Indicates electricity demand, clearing price Indicates the market clearing price. n Indicates the number of days;

[0015] S12, Unit Parameter Data: The parameter dataset for each unit is represented as follows:

[0016] ;

[0017] in, unit ID Indicates the unit number. Capacity Indicates the rated capacity of the unit. Utilization Hour This indicates the average annual operating hours of the power plant. Coal Consumption This indicates the amount of coal required to generate one kilowatt-hour of electricity. Power Consumption Rate It represents the percentage of electricity generated per unit of time in a power plant.

[0018] S13, Market Information Data for Forecast Date: Obtain market information data for the forecast date. It includes 24 data points per day.

[0019] Optionally, the market entity optimization in S2 includes:

[0020] S21, Probability Density Function Generation: A Gaussian Mixture Model (GMM) is used, assuming each data point in the dataset... Each data point is generated from one of K Gaussian distributions. The probability density function is expressed as:

[0021] ;

[0022] Where K is the number of Gaussian components. It is the mixing coefficient of the k-th Gaussian distribution;

[0023] S22, Parameter estimation: Solved using the Expectation-Maximization (EM) algorithm, specifically including:

[0024] Expectation step (E step): Based on the currently estimated parameters (mean) of the Gaussian mixture model Covariance matrix and mixing coefficient ), calculate each data point Posterior probability of belonging to each Gaussian distribution k , is represented as:

[0025] ;

[0026] Maximization step (M step): Using the calculated posterior probability Update the parameters of each Gaussian distribution, expressed as:

[0027] ;

[0028] ;

[0029] ;

[0030] Repeated iteration: Continuously update the parameters θ of the Gaussian mixture model until the log-likelihood function is obtained. The likelihood function is expressed as follows: The likelihood function terminates when the value of the expression converges or the predetermined number of iterations is reached:

[0031] ;

[0032] S23, Clustering effect evaluation: The fitting effect of the Gaussian mixture model is evaluated by calculating the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC).

[0033] Optionally, the setup of the spot market simulation environment in S3 includes:

[0034] S31, Clearing Rule Setting: All generator units are set to submit supply declarations in five segments, with the declared electricity volume evenly distributed in each segment. The supply information for different time segments is sorted by price, with each generator unit i's bid in segment s being [price missing]. The power generation is For each time period, according to The supply information and demand are sorted from low to high for trading, and transactions are conducted according to the price priority principle.

[0035] S32, Market Clearing: If the total transaction capacity is greater than or equal to the total demand D, the market is cleared and supply and demand are balanced. The bid price of the last unit that is traded is the clearing price. If the total bid capacity is less than the total demand, all supply is traded and the clearing price is the highest bid price. If there are multiple bids for the last unit that is traded, the electricity of these units needs to be redistributed proportionally, and the untraded units are added to the results list to obtain the final transaction result.

[0036] Optionally, the reward function system settings in S4 include:

[0037] S41, Price Deviation: The clearing price deviation at each time point is measured by calculating the squared relative error between the marginal price and the actual electricity price. Price deviation reward value The clearing price deviation at all time points is expressed as:

[0038] ;

[0039] ;

[0040] in, The market-clearing electricity price for each time period on a given day. This refers to the actual electricity price at each time point on that day. β These are the weighting coefficients;

[0041] S42, Contract Deviation: The annual cumulative electricity deviation is calculated by calculating the square of the relative error between the cumulative load rate and the target cumulative load rate. This is used to evaluate the impact of the agent's pricing strategy on transaction volume, and a time-sensitive weighting coefficient is set. To guide the agent's pricing strategy towards a predetermined target state, it is represented as:

[0042] ;

[0043] in, The cumulative load factor up to day t. It is the average load factor;

[0044] ;

[0045] ;

[0046] in, The initial weighting coefficients are used, and t is the current time step.

[0047] S43, Market Revenue: Based on the actual power generation of the generating unit, the market clearing price, and its power generation cost (coal consumption cost), calculate the total revenue and total cost of the market participants within a day, and subtract them to obtain the net revenue value. This is used to evaluate the economic performance of the agent's daily pricing strategy, and is expressed as:

[0048] ;

[0049] in, Let i be the amount of electricity generated by market entity i at time t. Let t be the marginal price (market clearing price) at time t. The coal consumption per unit of electricity generated by market entity i. For coal prices.

[0050] Optionally, the unit pricing strategy model in S5 includes:

[0051] S51, Agents and Environment: The bidding decision-making behavior of the power generation side is modeled as a Markov game process, each unit is defined as an agent with the ability to interact with the environment, the spot market is regarded as the external environment of the agent, and the MADDPG algorithm is used to simulate the dynamic behavior evolution process of the power generator.

[0052] S52, Observation Status (S): The observation status includes the transaction volume of unit i. G i Coal consumption The next day's load demand and market-cleared electricity prices ;

[0053] S53, Action Space (A): At each time t, the agent outputs an action value through the policy network based on its current local state. and the action vector It is divided into two parts, including the first half. and the second half With probability p(t) Add Gaussian noise n 1 N(0,σ 2 ) The processed action is With probability p(t) Add Gaussian noise n 2 N(0,σ 2 ) The processed action is , is represented as:

[0054] ;

[0055] ;

[0056] ;

[0057] S54, Reward Value (R): Reward Value Including price deviation Contract deviation and market returns R income ;

[0058] S55, Network Structure: The network structure includes a policy network (Actor) and a value network (Critic). The policy network (Actor) is a multilayer perceptron (MLP) structure, in which normalization is performed through the Softmax function to characterize the probability distribution of the unit's application volume, and regularization is used to simulate the agent's bidding behavior from low to high. The value network (Critic) adopts a multilayer perceptron structure.

[0059] Optionally, model training in S6 includes:

[0060] S61, Model Initialization: Initialize the policy network for each agent i. and value network Target policy network and target value network Initialize the experience replay pool D and motion noise p0, and set the total number of training rounds M and the data duration T;

[0061] S62, Autonomous Optimization: Each agent learns and iteratively optimizes in the continuous action space to gradually find the optimal strategy in order to maximize the cumulative reward;

[0062] S63, Training Data Storage and Model Dictionary Archiving: After each iteration, all interaction data generated in the current round is serialized and saved to disk. At the same time, after every 5 training rounds, the parameter dictionaries of the current policy network and value network are saved as checkpoint files.

[0063] Optionally, the autonomous optimization in S62 includes:

[0064] S621, Action Selection and Execution: For each training round (from 1 to M), initialize the environment and iterate the time steps. For each time step T (from 1 to the maximum round length), each agent performs action selection, environment interaction, and experience storage.

[0065] S622, Target Value Calculation: If D > Minimum Capacity, each agent will randomly select B samples from D and process them through the target value network. The target value was calculated. , is represented as:

[0066] ;

[0067] ;

[0068] ;

[0069] in, r i Let μ be the instantaneous reward for the i-th agent at the current time step, and μ be the discount factor. This represents the set of states of agent i, including the transaction volume Gi and coal consumption of each unit i. and the load demand for the next day and market-cleared electricity prices , The final set of actions for all subjects. d i This is the termination signal for the i-th agent;

[0070] S623, Network Parameter Update: By minimizing the loss function The value network is optimized to maximize its output (the Q-value of the action in the current state), which in turn optimizes the policy network. Regularization and action penalties are added to stabilize training, and an optimizer is used to adjust the policy network parameters. and value network parameters Backpropagation and updating are represented as follows:

[0071] ;

[0072] ;

[0073] Where λ is the weight of the squared penalty for the action, and α is the weight of the regularization term. It is the regularization loss of the policy network;

[0074] ;

[0075] ;

[0076] in, It is the learning rate of the policy network. It is the learning rate of the value network. This represents the gradient of the policy network loss function with respect to the policy network parameters. This represents the gradient of the value network loss function with respect to the value network parameters;

[0077] S624, Target Network Soft Update: For each agent, the target network parameters are softly updated, expressed as:

[0078] ;

[0079] ;

[0080] in, and These are the target policy network parameters and the target value network parameters, respectively, and τ is the update coefficient.

[0081] Optionally, the model evaluation and electricity price prediction in S7 include:

[0082] S71, Model Evaluation: Based on the data during the training process, evaluate the overall convergence performance of the generator bidding strategy model in the market environment;

[0083] S72, Predicting Electricity Prices: Based on the trained generator bidding strategy model, the input data for the period to be predicted is used as the input stream to generate a dynamic prediction sequence of spot electricity prices.

[0084] The beneficial effects of this invention are:

[0085] This invention effectively alleviates the dimensionality curse problem caused by high-dimensional data in the power market by using a Gaussian mixture model to reduce the dimensionality of market participants and combining the Akaike information criterion and the Bayesian information criterion for cluster optimization. This improves data processing efficiency and modeling accuracy. At the same time, by combining a marginal clearing mechanism to construct a spot market simulation environment, it can restore the decision-making logic of power generators under relatively realistic supply and demand game conditions.

[0086] This invention constructs a generator bidding strategy model based on the MADDPG algorithm and sets up a multi-dimensional reward function system that integrates price deviation, contract deviation, and market returns. It realizes a multi-agent autonomous learning architecture with centralized training and distributed execution, which enables the model to still have good convergence and generalization ability under limited information conditions. Finally, it can output dynamic electricity price prediction sequences suitable for short- and medium-term scenarios, significantly improving the timeliness and accuracy of prediction and meeting the intelligent needs of actual power market trading and dispatch. Attached Figure Description

[0087] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0088] Figure 1This is a schematic diagram of the prediction method flow according to an embodiment of the present invention;

[0089] Figure 2 This is a schematic diagram of the MADDPG algorithm flow according to an embodiment of the present invention;

[0090] Figure 3 This is a schematic diagram of the AIC and BIC evaluation metrics for Gaussian mixture clustering in an embodiment of the present invention;

[0091] Figure 4 This is a schematic diagram illustrating the changes in the cumulative experience values ​​of each intelligent agent on a certain day, according to an embodiment of the present invention.

[0092] Figure 5 This is a schematic diagram comparing the prediction effects of an embodiment of the present invention. Detailed Implementation

[0093] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. Those skilled in the art may employ other alternative methods to implement some well-known technologies; moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0094] like Figures 1-5 As shown, a method for forecasting short-to-medium-term electricity spot prices based on multi-agent simulation includes the following steps:

[0095] S1, Data Collection: Obtain historical market information data, unit parameter data, and forecast daily market information data;

[0096] S2, Market Entity Optimization: Gaussian Mixture Clustering (GMM) algorithm is used to reduce the dimensionality of market entities, and the number of entities is selected by calculating the Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC);

[0097] S3, build a spot market simulation environment: determine the price of electricity trading based on the marginal price mechanism and the market supply and demand relationship;

[0098] S4, Reward Function System Setting: A three-dimensional reward function system that integrates price deviation, contract deviation and market return is used to achieve multi-objective optimization by combining a dynamic weight adjustment mechanism;

[0099] S5, Constructing a generator unit bidding strategy model: Based on the reinforcement learning algorithm of Multi-Agent Deep Deterministic Policy Gradient (MADDPG), construct a generator bidding strategy model;

[0100] S6, Model Training: Train the multi-agent simulation model using reinforcement learning algorithms and implement the model's own autonomous optimization process;

[0101] S7, Model Evaluation and Electricity Price Forecasting: Evaluate the model's effectiveness based on empirical values ​​to achieve short- to medium-term electricity spot price forecasts.

[0102] The collected data in S1 includes:

[0103] S11, Historical Market Information Data: Daily historical market information includes 96 data points, forming the historical dataset. .in, date Indicates the date. time Indicates time, demand Indicates electricity demand, clearing price Indicates the market clearing price. n Indicates the number of days;

[0104] S12, Unit Parameter Data: The information for each unit includes the following dimensions, constituting the unit's technical parameter dataset. .in, unit ID Indicates the unit number. Capacity Indicates the rated capacity of the unit. Utilization Hour This indicates the average annual operating hours of the power plant. Coal Consumption This indicates the amount of coal required to generate one kilowatt-hour of electricity. Power Consumption Rate It represents the percentage of electricity generated per unit of time in a power plant.

[0105] S13, Market Information Data for Forecast Date: Obtain market information data for the forecast date. It includes 24 data points per day.

[0106] The optimization of market participants in S2 includes:

[0107] S21, Probability Density Function Generation: The GMM algorithm assumes that each data point in the dataset... Each data point is generated from one of K Gaussian distributions. The probability density function is expressed as:

[0108] ;

[0109] Where K is the number of Gaussian components. It is the mixing coefficient of the k-th Gaussian distribution, and satisfies ;

[0110] S22, Parameter Estimation: Solved using the Expectation-Maximization (EM) algorithm. The Expectation Step (E-step) is based on the currently estimated model parameters (mean). Covariance matrix and mixing coefficient ), calculate each data point Response degree belonging to each Gaussian distribution k , is represented as:

[0111] ;

[0112] The maximization step (M-step) utilizes the calculated posterior probability. Update the parameters of each Gaussian distribution. The specific update formula is as follows:

[0113] ;

[0114] ;

[0115] ;

[0116] Repeat the EM step, iteratively updating the model parameters θ until the log-likelihood function is reached. When the value of converges, or the predetermined number of iterations is reached, the likelihood function is expressed as:

[0117] ;

[0118] S23, Model Evaluation: The model's fit is evaluated by calculating the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC). A smaller AIC value indicates that the model has achieved a better balance between fitting the data and model complexity; similarly, a smaller BIC value indicates a better model.

[0119] The spot market simulation environment in S3 includes:

[0120] S31, Clearing rule setting: Assume all generating units submit supply bids in five segments, with the electricity volume in each segment evenly distributed. The supply information for different time segments is sorted by price, with each unit i's bid in bidding segment s being... The power generation is For each time period, according to The supply information and demand are sorted from low to high for trading, and transactions are conducted according to the price priority principle.

[0121] S32, Market Clearing: If the total traded capacity is greater than or equal to the total demand D, the market is cleared to achieve supply and demand equilibrium, and the bid price of the last traded unit is the clearing price. If the total bid capacity is less than the total demand, all supply is traded, and the clearing price is the highest bid price. If multiple units have the same bid for the last traded unit, the electricity volume of these units needs to be redistributed proportionally, and the untraded units are added to the results list to obtain the final trading result.

[0122] The reward function system settings in S4 include:

[0123] S41, Price Deviation: Clearing price deviation at each time point Price deviation bonus is measured by calculating the squared relative error between the marginal price and the actual electricity price. The clearing price deviation at all time points is expressed as:

[0124] ;

[0125] ;

[0126] in The market-clearing electricity price, i.e., the marginal price, for each time point on a given day. This refers to the actual electricity price at each time point on that day. β This refers to the weighting coefficient. (Based on the total reward) This can encourage agents to quote prices close to market-clearing electricity prices, thereby reducing price discrepancies and bringing them closer to the true value.

[0127] S42, Contract Deviation: Specifically refers to the annual cumulative electricity deviation. This is used to evaluate the impact of the agent's pricing strategy on transaction volume. This invention calculates the reward by calculating the squared relative error between the cumulative load rate and the target cumulative load rate, expressed as:

[0128] ;

[0129] in The cumulative load factor up to day t. This is the average load factor. Additionally, to reflect the time-series progression of deviation penalties, a time-sensitive weighting coefficient is set. To guide the agent's pricing strategy to more closely approximate the predetermined target state, the calculation formula is as follows:

[0130] ;

[0131] ;

[0132] in The initial weight coefficients are used, and t is the current time step.

[0133] S43, Market Returns: This is an economic reward indicator; the reward value... To evaluate the economic performance of an agent's daily bidding strategy, based on the unit's actual power generation, market clearing price, and its power generation cost (coal consumption cost), the total revenue and total cost of the market participant within a day are calculated, and the net profit value is obtained by subtracting them, expressed as:

[0134] ;

[0135] in Let i be the amount of electricity generated by market entity i at time t. Let t be the marginal price (market clearing price) at time t. The coal consumption per unit of electricity generated by market entity i. The market return reward value directly reflects the economic efficiency and profitability of the agent's pricing strategy, and is a key indicator for evaluating the quality of the strategy. (This refers to the coal price.)

[0136] The generator bidding strategy model in S5 includes:

[0137] S51, Agents and Environment: This invention proposes a bidding strategy model for power generators based on multi-agent deep reinforcement learning. The bidding decision-making behavior on the generation side is modeled as a Markov game process. Each generating unit is defined as an agent capable of interacting with the environment, the spot market is considered the external environment of the agents, and the MADDPG algorithm is used to simulate the dynamic behavioral evolution process of power generators.

[0138] S52, Observation State (S): refers to the information received by the agent from the environment. The observation state of this invention consists of four parts, namely, the transaction volume of group i. G i Coal consumption The next day's load demand and market-cleared electricity prices The unit's transaction volume refers to the actual electricity volume transacted by the unit in the previous market clearing cycle. This information helps the unit predict its production efficiency and revenue, assists the agent in understanding its market performance, and adjusts its pricing strategy accordingly. Coal consumption refers to the amount of coal consumed by the unit during electricity production. In this embodiment, it can be directly read from the data and is a key indicator for measuring the unit's operating costs. The next day's load demand is the electricity demand for the next day in the data. Understanding this demand helps the unit clarify the market supply and demand relationship, predict market conditions, and thus formulate a more effective pricing strategy. Finally, the market clearing price, which is the indicator of interest in this invention, is a key signal in the electricity market, helping the agent understand the market pricing mechanism and adjust its pricing accordingly to maximize profits. These observations will be input into the neural network of each agent as key information for its interaction with the environment, becoming part of the learning process, enabling the agent to more comprehensively understand the market state and the impact of its own behavior during interaction with the environment.

[0139] S53, Action Space (A): Specifically refers to the set of all possible actions that an agent can perform. MADDPG is an extension of the DDPG algorithm in a multi-agent environment, aiming to solve the reinforcement learning challenges in multi-agent scenarios. This algorithm employs a "centralized training, distributed execution" (DTDE) paradigm for training. DTDE means that all agents share a centralized Critic network. During training, the Critic network can access information from all agents and provide guidance to each agent's Actor network. However, during execution, each agent's Actor network acts completely independently, achieving decentralized execution. Specifically, at each time t, the agent, based on its current local state, can directly output action values ​​through the policy network. , is represented as:

[0140] ;

[0141] Among them, the action vector Processed in two parts Indicates the first half. This is the second half. Time t is equivalent to the current step number, and t0 is the step number from which the noise probability adjustment begins. The probability p(t) is used to... Add Gaussian noise n 1 N(0,σ 2 ) The processed action is Similarly, with probability p(t) for... Add Gaussian noise n 2 N(0,σ 2 ) The processed action is , is represented as:

[0142] ;

[0143] The agent performs a new action Interacting with the environment is represented as:

[0144] ;

[0145] S54, Reward Value (R): Refers to the feedback the agent receives from the environment, used to evaluate the quality of the agent's performance of an action. The reward value in this paper... Set as price deviation Contract deviation and market returns R income The three parts are integrated. Each interaction between the agent and the environment will generate a termination signal d.i (A value of 1 indicates a terminated state, and a value of 0 indicates a non-terminated state), Reward and the state at the next moment To improve training stability and efficiency, MADDPG employs an experience replay mechanism. This mechanism records the agent's state, action, reward, next state, and termination flag for each interaction. The experience will be stored in the experience replay pool D. During training, a batch of experience will be randomly drawn from this buffer to update the network.

[0146] S55, Network Structure: This invention discloses a generator unit bidding strategy model based on the MADDPG reinforcement learning algorithm. This algorithm equips each agent with an independent Actor-Critic framework. The policy network (Actor) determines the agent's action value, i.e., the electricity quantity and price declared by the generator unit, while the value network (Critic) evaluates the action and provides the corresponding reward.

[0147] The Actor network is a typical multilayer perceptron (MLP) architecture, containing two fully connected layers to output the probability distribution of actions. This network introduces non-linear features through the ReLU activation function and applies scaling factors to adjust the range of activation values. To achieve the agent's target action, the output is processed in two parts: the first part is normalized using the Softmax function to ensure the sum of all elements is 1, and a minimum constraint is applied to characterize the probability distribution of the crew's declaration volume; the second part is regularized to ensure that each output value is distinct and arranged in ascending order, thus simulating the agent's bidding behavior from low to high. The Critic network also employs a multilayer perceptron structure, containing multiple hidden layers and one output layer. Its key feature is that as the network deepens, the number of neurons in the hidden layers decreases layer by layer until an output layer is reached, converting the input state-action pair into a value prediction. This progressively decreasing neuron configuration allows the Critic network to learn complex function mappings, thereby accurately evaluating the value of input data and providing a reliable basis for decision-making in the reinforcement learning process.

[0148] like Figure 2 As shown, model training in S6 includes:

[0149] S61, Model Initialization: Initialize the Actor network for each agent i. and Critic Network Target Actor Network and target Critic network Initialize the experience replay pool D and motion noise p0, and set the total number of training rounds M and the data duration T;

[0150] S62, Autonomous Optimization: Each agent learns and iteratively optimizes within a continuous action space to gradually find the optimal strategy, maximizing long-term cumulative rewards. This autonomous optimization process requires no human intervention and is entirely driven by the algorithm itself.

[0151] The autonomous optimization process in S62 includes:

[0152] S621, Action Selection and Execution: For each training round (from 1 to M), initialize the environment and iterate through the time steps. For each time step T (from 1 to the maximum round length), each agent performs action selection, environment interaction, and experience storage.

[0153] S622, Target Value Calculation: If D > Minimum Capacity, each agent will randomly select B samples from D and pass them through the target network. The target value was calculated. , is represented as:

[0154] ;

[0155] ;

[0156] ;

[0157] Where, r i It is the instantaneous reward of the i-th agent at the current time step, and μ is the discount factor. This represents the set of states of agent i, including the transaction volume Gi and coal consumption of each unit i. and the load demand for the next day and market-cleared electricity prices , The final set of actions for all subjects. d i This is the termination signal for the i-th agent.

[0158] S623, Network Parameter Update: By minimizing the loss function The optimized Critic network is represented as:

[0159] ;

[0160] The policy network is optimized by maximizing the output of the Critic network (the Q-value of the action in the current state), while regularization and action penalties are added to stabilize training, as follows:

[0161] ;

[0162] Where λ is the weight of the squared penalty for the action, and α is the weight of the regularization term. This is the regularization loss of the policy network. The optimizer is used to adjust the parameters of the Actor network. and Critic network parameters Backpropagation and updating are represented as follows:

[0163] ;

[0164] ;

[0165] in, It is the learning rate of the Actor network. It is the learning rate of the Critic network. This represents the gradient of the Actor loss function with respect to the Actor network parameters. This represents the gradient of the Critic loss function with respect to the Critic network parameters.

[0166] S624, Target Network Soft Update: For each agent, the target network parameters are softly updated, expressed as:

[0167] ;

[0168] ;

[0169] in, and These are the target Actor network parameters and the target Critic network parameters, respectively, and τ is the update coefficient;

[0170] S63, Training Data Storage and Model Dictionary Archiving: After each iteration, all interaction data generated in the current round is serialized and saved to disk. At the same time, after every 5 training rounds, the parameter dictionaries of the current Actor network and Critic network are saved as checkpoint files to support rapid recovery or policy comparison after training interruption.

[0171] The model evaluation and electricity price forecasting in S7 include:

[0172] S71, Model Evaluation: Based on data from the training process, evaluate the overall convergence performance of the deep reinforcement learning model in a market environment. Quantitative analysis and evaluation are conducted on the historical reward values ​​and bidding strategies of typical market participants.

[0173] S72, Predicting Electricity Prices: Load the trained deep reinforcement learning model, take the input data of the period to be predicted as the input stream, and generate a dynamic prediction sequence of spot electricity prices through model strategy deduction.

[0174] In this embodiment, the historical market information of a certain province's spot market is shown in Table 1. The start and end times of each day are 00:15 and 24:00, with an interval of 15 minutes, and there are 96 data points per day.

[0175] Table 1 Historical Market Data

[0176]

[0177] Data analysis revealed the upper and lower limits of generator unit pricing. Therefore, when building the simulation environment, a price cap was set for the electricity spot market, specifying a clearing price upper limit of 1800 yuan / MWH and a lower limit of -50 yuan / MWH. Although the market clearing frequency was set to once every 15 minutes, the clearing price remained fixed within one hour. Therefore, to improve the model performance of the simulation environment, the 96 daily data points were converted into 24-point data, that is, the arithmetic average of the four 15-minute data points of each hour was taken as the representative value for that hour.

[0178] The adjusted data format is shown in the table below. The start and end times for each day are 0:00 and 23:00, with an interval of 1 hour, and 24 data points per day.

[0179] Table 2 Processed Historical Market Data

[0180]

[0181] In addition, it includes technical parameters for 549 generator sets, as shown in Table 3, covering key indicators such as unit ID, type, installed capacity, coal consumption for power supply, and utilization hours. In multi-agent decision-making simulations, directly including all agents faces three major challenges: excessive data scale, reduced computational efficiency, and difficulty in model generalization. To effectively address these issues, this invention introduces a Gaussian mixture clustering algorithm to reduce the dimensionality and filter market agents. The model is evaluated by calculating the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC). Lower AIC and BIC values ​​indicate a higher goodness of fit and lower model complexity, representing the optimal number of clusters. Figure 3 As shown, 42 cluster centers were ultimately selected as representatives of typical market entities, and a multi-agent simulation model was constructed accordingly.

[0182] Table 3 Market Participant Information

[0183]

[0184] The model was trained using relevant historical data from 12 months of 2022 as the environment, with a clearing frequency of 1 hour, a training interval of 24 hours per day, and an algorithm training round of 200 rounds.

[0185] like Figure 4 The graph illustrates the changes in the cumulative experience value of each agent on a given day during model training. Each subgraph corresponds to one agent, with the horizontal axis representing the number of training epochs and the vertical axis representing the cumulative reward value. It can be observed that in the early stages of reinforcement learning training, the cumulative reward values ​​of the agents mostly exhibit significant fluctuations, as they actively try different strategies to gain experience. As training progresses, the agents gradually reduce this exploratory behavior and instead apply more of the strategies they have already learned, leading to a decrease in the volatility of reward values ​​and a more stable trend. After approximately 50 epochs of learning, the agents' exploratory behavior further decreases, and they begin to rely more on accumulated bidding experience, causing their reward values ​​to continue to increase. This upward trend continues until approximately 120 epochs, at which point the cumulative reward values ​​of most agents show signs of convergence, and the fluctuation range further decreases. This indicates that the learning process of the agents has reached a relatively stable state, and their bidding strategies have become more mature and effective.

[0186] according to Figure 4 As a result, it can be observed that after a period of learning, the agents' experience values ​​mostly tend to stabilize. This indicates that most agents have found relatively effective strategies and are able to stably apply these strategies to acquire experience values. Therefore, the model's training results are quite satisfactory and can be used for in-depth predictive analysis.

[0187] This invention uses the aforementioned model to predict the clearing electricity price for 744 time points over the next 31 days based on historical market clearing prices. The predictions are compared with those of Long Short-Term Memory (LSTM), Support Vector Machine Regression (SVR), and Random Forest (RF) models. The mean absolute error (MAE), mean squared error (MSE), and mean absolute percentage error (MAPE) values ​​are shown in Table 4. Lower MAE, MSE, and MAPE indicate better predictive performance. The results show that the proposed method exhibits excellent overall performance across all evaluation metrics, indicating a smaller deviation between its predicted results and the actual values, and stronger stability, demonstrating its advantages in practical prediction tasks.

[0188] In addition, data from four consecutive days, from day 26 to day 29 of the forecast phase, were specifically selected and expanded to a format with 96 time points to match the temporal resolution of the original data. The comparison results are as follows: Figure 5 As shown. For this scenario, the multi-agent simulation model using the MADDPG algorithm can more sensitively capture the peak, trough, and flat changes in electricity prices. It can not only better follow subtle changes in electricity prices and respond to price fluctuations in a timely manner, but also effectively avoid situations where the predicted values ​​lag or deviate from the actual fluctuations.

[0189] Table 4 Comparison of Model Prediction Results

[0190]

[0191] In summary, the application of this patented technology significantly improves forecasting performance on short- to medium-term time scales compared to other traditional methods (LSTM, SVR, RF), especially under extreme electricity price conditions. This provides a scientific basis for market participants to make decisions in the electricity spot market, thereby increasing market returns.

[0192] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0193] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for short and medium term electricity spot price forecasting based on multi-agent simulation, characterized in that, Comprising the following steps: S1, data collection: obtain historical market information data, unit parameter data and predicted day market information data; S2, market subject optimization: Gaussian mixture model is used for feature dimension reduction of market subjects, and Akaike information criterion and Bayesian information criterion are used to select the number of subjects; S3, build a spot market simulation environment: according to the marginal price mechanism, the price of power transaction is determined through the market supply and demand relationship; S4, reward function system setting: through the three-dimensional reward function system of price deviation, contract deviation and market income, combined with the dynamic weight adjustment mechanism, multi-objective optimization is realized; S5, constructing a unit bidding strategy model: based on the multi-agent deep deterministic policy gradient reinforcement learning algorithm, a power supplier bidding strategy model is constructed; S6, model training: the power supplier bidding strategy model is trained, and the self-optimization process of the power supplier bidding strategy model is realized; S7, model evaluation and electricity price prediction: the effect of the power supplier bidding strategy model is evaluated according to the experience value, and the short-term time scale power spot price prediction is realized; The reward function system setting in S4 comprises: S41, price deviation: the clearing price deviation of each time node is measured by calculating the relative error square between the marginal price and the actual electricity price , price deviation reward value is the clearing price deviation of all time nodes, expressed as: ； ； wherein, is the market clearing price for each time of day, is the actual price for each time of day, β is a weight coefficient; S42, contract deviation: calculate the annual cumulative electricity quantity deviation by calculating the relative error square between the cumulative load rate and the target cumulative load rate , for evaluating the influence of the bidding strategy of the agent on the transaction volume, and setting a time-sensitive weight coefficient , to guide the agent's bidding strategy to approach the predetermined target state, expressed as: ； wherein, is the cumulative load rate up to day t, is the average load rate; ； ； wherein, wherein, W0denotes the initial weight coefficient, t is the current time step. S43, market revenue: based on the actual generation of the unit, the market clearing price and its generation cost, the total revenue and total cost of the market subject in a day are calculated, and the net income value is obtained by subtraction The economic performance of the daily bidding strategy of the agent is evaluated, which is represented as: ； wherein, is the generation of market agent i at time t, is the nodal marginal price at time t, is the coal consumption of market agent i per unit of generation, is the coal price.

2. The method according to claim 1, wherein, The data collection in S1 comprises: S11, historical market information data: daily historical market information includes 96 data, which constitutes a historical data set wherein, date denotes date, time denotes time, demand denotes power demand, clearing price denotes market clearing price, n denotes day number; S12, unit parameter data: the parameter data set of each unit is represented as: ； wherein, unit ID represents the unit number, Capacity represents the rated capacity of the unit, Utilization Hour represents the annual average operating hours of the power plant, Coal Consumption represents the amount of coal consumed per unit of electricity generated, Power Consumption Rate represents the percentage of electricity generated per unit of time; S13, predicting day market information data: obtaining market information data of the predicting day Each day includes 24 data.

3. The method according to claim 2, wherein the method is characterized by, The market subject optimization in S2 comprises: S21, Probability Density Function Generation: Adopting Gaussian Mixture Model, assuming that each data point in the data set is generated by one of the K Gaussian distributions, obtaining the probability density function of the data point, denoted as: S22, Probability Density Function Generation: Adopting Gaussian Mixture Model, assuming that each data point in the data set is generated by one of the K Gaussian distributions, obtaining the probability density function of the data point, denoted as: S22, Probability Density Function Generation: Adopting Gaussian Mixture Model, assuming ； where K is the number of Gaussian components, is the mixing coefficient of the kth Gaussian distribution; S22, parameter estimation: expectation maximization algorithm is used for solution, specifically comprising: Expected step: Compute each data point Posterior probability of belonging to each Gaussian distribution k is denoted by: ； Maximization step: Update the parameters of each Gaussian distribution using the computed posterior probabilities , denoted as: ； ； ； repeated iterations: iteratively update the Gaussian mixture model parameters θ until the value of the log-likelihood function converges or a predetermined number of iterations is reached, the likelihood function is represented as: ； S23, clustering effect evaluation: the fitting effect of Gaussian mixture model is evaluated by calculating Akaike information criterion and Bayesian information criterion.

4. The method according to claim 3, wherein the method is characterized by, The building of spot market simulation environment in S3 comprises: S31, clearing rule setting: set all generators to five segments for reporting supply, evenly distribute the reported power of each segment, take one day as a unit, one hour as a segment, sort the supply information of different time segments according to price, and the offer of each unit i in the offer segment s is , and the power generation is For each time segment, according to , the supply information and demand quantity sorted from low to high are traded, and the transaction follows the principle of price priority; S32, market clearing: if the total transaction capacity is greater than or equal to the total demand D, that is, the market clearing state of supply and demand balance is reached, the bid of the last transaction unit is the clearing price, if the total declared capacity is less than the total demand, all supply capacity is transacted, and the clearing price is the highest bid price, if there are multiple same bids for the last transaction unit, the power of these units needs to be proportionally redistributed, and then the untransacted units are supplemented to the result list to obtain the final transaction result.

5. The method according to claim 4, wherein the method is characterized by, The construction of unit bidding strategy model in S5 comprises: S51, agent and environment: the bidding decision behavior of the power generation side is modeled as a Markov game process, each unit is defined as an agent with the ability to interact with the environment, the spot market is regarded as the external environment of the agent, and the MADDPG algorithm is used to simulate the dynamic behavior evolution process of the power supplier; S52, observed state: the observed state includes the volume of trades for unit i G i , coal consumption , load demand for the next day and market clearing price ; S53, action space: at each time t, the agent outputs an action value through the policy network according to its current local state , and divides the action vector into two parts, including the first half and the second half , adds Gaussian noise to with a probability p(t) n 1 N(0,σ 2 ) , and the processed action is , adds Gaussian noise to with a probability p(t) n 2 N(0,σ 2 ) , and the processed action is , which is represented as: ； ； ； S54, reward value: reward value including price deviation , contract deviation and market return R income ; S55, network structure: the network structure comprises a policy network and a value network, the policy network is a multilayer perceptron structure, wherein the Softmax function is used for normalization to represent the probability distribution of unit declared capacity, and the value network adopts a multilayer perceptron structure.

6. The method according to claim 5, wherein the method is characterized by, The model training in S6 comprises: S61, model initialization: initialize the policy network for each agent i and the value network , the target policy network and the target value network , initialize the experience replay pool D and the action noise p 0, and set the total number of training rounds M and the time length of data T; S62, autonomous optimization: each agent finds the optimal strategy by self-learning and iterative optimization in the continuous action space to maximize the cumulative reward; S63, training data storage and model dictionary archiving: after each iteration, all interaction data generated in the current round are serialized and saved to the disk, and the parameter dictionary of the current strategy network and value network is saved as a checkpoint file after completing 5 training rounds.

7. The method according to claim 6, wherein the method is characterized by, The autonomous optimization in S62 includes: S621, action selection and execution: for each training round, the environment is initialized and the time step iteration is performed, and for each agent at each time step T, action selection, environment interaction, and experience storage are performed; S622, target value calculation: if D> minimum capacity, each agent will randomly draw B samples from D, and the target value network The target value is calculated , which is expressed as: ； ； ； where ri is the immediate reward of the ith agent at the current time step, μ is the discount factor, denotes the state set of agent i, including the traded volume Gi and coal consumption of each unit i and the load demand of the next day and the market clearing price , is the final action set of all agents, d i is the termination signal of the ith agent; S623, network parameter update: by minimizing the loss function Optimizing the value network to maximize the output of the value network to optimize the policy network while adding regularization and action penalties to stabilize training and using an optimizer to update the policy network parameters and the value network parameters backpropagation and update, represented as: ； ； where λ is the weight of the action square penalty, and a is the weight of the regularization term, is the regularization loss of the policy network; ； ； wherein, is a learning rate for the policy network, is a learning rate for the value network, denotes the gradient of the policy network loss function with respect to the policy network parameters, denotes the gradient of the value network loss function with respect to the value network parameters; S624, target network soft update: for each agent, the target network parameters are soft updated, denoted as: ； ； wherein, and are target policy network parameters and target value network parameters, respectively, and τ is an update coefficient.

8. The method according to claim 7, wherein the method is characterized by, The model evaluation and electricity price prediction in S7 include: S71, model evaluation: based on the data in the training process, the overall convergence performance of the power supplier bidding strategy model in the market environment is evaluated; S72, prediction of electricity price: based on the trained power supplier bidding strategy model, the input data of the to-be-predicted period is taken as the input stream to generate a dynamic prediction sequence of the spot electricity price.

Images