Decision-making method and device for participation of building load aggregator in power market bidding

Abstract

The invention discloses a building load aggregator participating power market bidding decision-making method and device. The method comprises the following steps: generating a simulation data set through a time generative adversarial network model; constructing a probability distribution prediction model based on a Bayesian network; training the probability distribution prediction model through the simulation data set to obtain a trained probability distribution prediction model; inputting the power market state data into the trained probability distribution prediction model for processing to obtain opponent quotation distribution data; based on a depth deterministic strategy gradient algorithm, constructing an Actor-Critic neural network model; the Actor-Critic neural network model is trained by setting a reinforcement learning tool box, and a trained Actor-Critic neural network model is obtained; and inputting opponent quotation distribution data into the trained Actor-Critic neural network model for processing, and obtaining an optimal bidding quantity price combination. According to the method, the electricity market environment of various factors can be comprehensively considered, and the optimal bidding quantity price combination is provided for the building load aggregator.

Description

Technical Field

[0001] The present invention relates to the technical field of power market bidding decision-making, and in particular to a method and device for building load aggregators to participate in power market bidding decision-making. Background Art

[0002] With the continuous growth of global energy demand and the advancement of the transformation of power systems to intelligent and renewable energy, the structure and operation mechanism of the power market have become increasingly complex. In this context, the participants in the power market are not limited to traditional power generation companies and power grid companies. More and more distributed energy resources and demand-side management participants have begun to enter the market, forming a diversified market ecosystem.

[0003] As an important role on the demand side, building load aggregators are responsible for integrating the electricity demand of multiple building units and achieving effective response to the electricity market through optimized scheduling and control. By participating in the electricity energy market and demand response projects, building load aggregators can not only improve the overall efficiency and stability of the power system, but also create economic benefits for themselves and the participating building units. However, how to formulate a scientific and effective bidding strategy for quantity and quotation has become the key for building load aggregators to gain a competitive advantage in the complex electricity market. The bidding strategies that existing research mainly focuses on are bidding quantity without quotation and quoting without bidding quantity. With the deepening of electricity market reform, quoting quantity will become the mainstream. Existing research has considered too few factors when characterizing the behavior of opponents, resulting in an inaccurate bidding strategy training environment.

[0004] Therefore, how to invent a comprehensive electricity market bidding decision-making method that can comprehensively consider various factors and improve the bidding strategy training environment has become an urgent problem to be solved. Summary of the invention

[0005] To this end, the present invention provides a method and device for building load aggregators to participate in power market bidding decision-making, which can comprehensively consider the power market environment of various factors and provide the building load aggregators with the best bidding quantity and price combination.

[0006] In order to achieve the above object, the present invention provides the following technical solution: a decision-making method for a building load aggregator to participate in a power market bid, comprising:

[0007] The historical transaction data, demand data and price fluctuation data of the electricity market are modeled through the time-generated adversarial network model to generate a simulated data set;

[0008] Based on Bayesian network, construct probability distribution prediction model;

[0009] Training the probability distribution prediction model using the simulated data set to obtain a trained probability distribution prediction model;

[0010] Inputting the power market status data into the trained probability distribution prediction model for processing to obtain the opponent's quotation distribution data;

[0011] Based on the deep deterministic policy gradient algorithm, an Actor-Critic neural network model is constructed; the Actor-Critic neural network model is trained by setting a reinforcement learning toolbox to obtain the trained Actor-Critic neural network model;

[0012] The opponent's bid distribution data is input into the trained Actor-Critic neural network model for processing to obtain the optimal bid quantity and price combination.

[0013] As a preferred solution for a building load aggregator to participate in a power market bidding decision-making method, the steps of generating the simulation data set through the time generative adversarial network model are:

[0014] Pre-process the collected raw data by setting data cleaning strategy and normalization processing strategy;

[0015] Constructing the temporal generative adversarial network model, and obtaining the trained temporal generative adversarial network model through training; generating time series data through the trained temporal generative adversarial network model;

[0016] The time series data is subjected to data denormalization processing and data verification processing to obtain the simulated data set.

[0017] As a preferred solution for a building load aggregator to participate in power market bidding decision-making method, the nodes in the probability distribution prediction model include: environmental factor nodes, market factor nodes, risk preference nodes, cost structure nodes, load forecast nodes and quotation strategy nodes; the nodes are associated with each other through directed edges.

[0018] As a preferred solution for a building load aggregator to participate in the power market bidding decision-making method, the Actor-Critic neural network model includes an Actor neural network and a Critic neural network; the Actor neural network is used to receive the state input of the current environment and generate corresponding action output; the Critic neural network is used to receive the joint input of the state and action, evaluate the quality of the generated action and output the corresponding Q value.

[0019] As a preferred solution for a building load aggregator to participate in the power market bidding decision-making method, in the process of training the Actor-Critic neural network model by using the set reinforcement learning toolbox, the Actor-Critic neural network model is trained by using the DDPG deep reinforcement learning toolbox in MATLAB and the GPU; the training steps are:

[0020] Inputting the initial demand data and the counterparty quotation distribution data into the Actor-Critic neural network model;

[0021] The Actor network in the Actor-Critic neural network model processes the initial demand data and the opponent's quotation distribution data to generate an action output;

[0022] The Critic network in the Actor-Critic neural network model performs quality assessment on the action output and outputs a Q value;

[0023] The Actor network updates the parameters of the Actor-Critic neural network model based on the policy gradient according to the Q value, and obtains a reward through environmental function calculation;

[0024] When the reward function converges, the training is stopped to obtain the trained Actor-Critic neural network model.

[0025] The present invention also provides a decision-making device for building load aggregators to participate in power market bidding, based on the above decision-making method for building load aggregators to participate in power market bidding, comprising:

[0026] A simulation data set generation module is used to model the historical transaction data, demand data and price fluctuation data of the power market through a time-generated adversarial network model to generate a simulation data set;

[0027] Probability distribution prediction model building module, used to build a probability distribution prediction model based on Bayesian network;

[0028] A probability distribution prediction model training module, used to train the probability distribution prediction model through the simulated data set to obtain the trained probability distribution prediction model;

[0029] A probability distribution prediction model prediction processing module is used to input the power market status data into the trained probability distribution prediction model for processing to obtain the opponent's quotation distribution data;

[0030] Actor-Critic neural network model construction and training module, used to construct an Actor-Critic neural network model based on a deep deterministic policy gradient algorithm; train the Actor-Critic neural network model by setting a reinforcement learning toolbox to obtain the trained Actor-Critic neural network model;

[0031] The Actor-Critic neural network model processing module is used to input the opponent's quotation distribution data into the trained Actor-Critic neural network model for processing to obtain the optimal bidding quantity and price combination.

[0032] As a preferred solution for a building load aggregator to participate in a power market bidding decision-making device, in the simulation data set generation module, the submodule for generating the simulation data set includes:

[0033] The raw data preprocessing submodule is used to preprocess the collected raw data by setting data cleaning strategy and normalization processing strategy;

[0034] A temporal generative adversarial network model construction and processing submodule is used to construct the temporal generative adversarial network model and obtain the trained temporal generative adversarial network model through training; and generate time series data through the trained temporal generative adversarial network model;

[0035] The simulation data set generation submodule is used to perform data denormalization and data verification processing on the time series data to obtain the simulation data set.

[0036] As a preferred solution for a building load aggregator to participate in the power market bidding decision-making device, in the probability distribution prediction model construction module, the nodes in the probability distribution prediction model include: environmental factor nodes, market factor nodes, risk preference nodes, cost structure nodes, load forecast nodes and quotation strategy nodes; each node is associated through directed edges.

[0037] As a preferred solution for a building load aggregator to participate in the power market bidding decision-making device, in the Actor-Critic neural network model construction and training module, the Actor-Critic neural network model includes an Actor neural network and a Critic neural network; the Actor neural network is used to receive the state input of the current environment and generate corresponding action output; the Critic neural network is used to receive the joint input of the state and action, evaluate the quality of the generated action and output the corresponding Q value.

[0038] As a preferred solution for a building load aggregator to participate in the power market bidding decision-making device, in the Actor-Critic neural network model construction and training module, the Actor-Critic neural network model is trained by using the GPU through the DDPG deep reinforcement learning toolbox in MATLAB; the submodules for training the Actor-Critic neural network model include:

[0039] A data input submodule, used to input the initial demand data and the opponent's quotation distribution data into the Actor-Critic neural network model;

[0040] An action output generation submodule, used for the Actor network in the Actor-Critic neural network model of the Actor network to process the initial demand data and the opponent's quotation distribution data to generate an action output;

[0041] An action output quality assessment submodule, used for the Critic network in the Actor-Critic neural network model to perform quality assessment on the action output and output a Q value;

[0042] A parameter updating submodule, used for the Actor network to update the parameters of the Actor-Critic neural network model based on the policy gradient according to the Q value, and obtain rewards through environmental function calculation;

[0043] The trained Actor-Critic neural network model acquisition submodule is used to stop training when the reward function converges and obtain the trained Actor-Critic neural network model.

[0044] The present invention has the following advantages: the present invention models the historical transaction data, demand data and price fluctuation data of the power market through a time generative adversarial network model to generate a simulation data set; based on a Bayesian network, a probability distribution prediction model is constructed; the probability distribution prediction model is trained through the simulation data set to obtain the trained probability distribution prediction model; the power market state data is input into the trained probability distribution prediction model for processing to obtain the opponent's quotation distribution data; based on a deep deterministic policy gradient algorithm, an Actor-Critic neural network model is constructed; the Actor-Critic neural network model is trained by setting a reinforcement learning toolbox to obtain the trained Actor-Critic neural network model; the opponent's quotation distribution data is input into the trained Actor-Critic neural network model for processing to obtain the optimal bidding quantity and price combination. The present invention can comprehensively consider the power market environment of various factors and provide a method for the optimal bidding quantity and price combination for building load aggregators. The present invention generates market data based on a time generative adversarial network (TimeGAN), constructs a Bayesian network for the power market, learns the probability distribution of opponent behavior, and optimizes the quotation and quotation decision through deep reinforcement learning (DDPG). The intelligent agent combines market demand, price fluctuations and other factors to generate the optimal bidding strategy to balance benefits and risks and maximize long-term profits. The system improves the accuracy and adaptability of power market bidding decisions through simulating market environments and strengthening learning training, and promotes the intelligent development of the market. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0046] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0047] Figure 1 A schematic diagram of a process flow of a decision-making method for a building load aggregator to participate in a power market bidding provided in Embodiment 1 of the present invention;

[0048] Figure 2A schematic diagram of comparison between samples generated and historical training data in a decision-making method for building load aggregators to participate in power market bidding provided in Example 1 of the present invention;

[0049] Figure 3 A schematic diagram of the structure of a Bayesian network in a decision-making method for building load aggregators to participate in power market bidding provided in Example 1 of the present invention;

[0050] Figure 4 A schematic diagram of the training results of calling the DDPG toolbox in MATLAB in a decision-making method for building load aggregators to participate in power market bidding provided in Example 1 of the present invention;

[0051] Figure 5 This is a schematic diagram of the architecture of a building load aggregator participating in power market bidding decision-making device provided in Example 2 of the present invention. DETAILED DESCRIPTION

[0052] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] Example 1

[0054] See also Figure 1 Embodiment 1 of the present invention provides a decision-making method for a building load aggregator to participate in a power market bidding, comprising the following steps:

[0055] S1. Model the historical transaction data, demand data and price fluctuation data of the electricity market through a time-generated adversarial network model to generate a simulation data set;

[0056] S2. Construct a probability distribution prediction model based on Bayesian network;

[0057] S3, training the probability distribution prediction model using the simulated data set to obtain the trained probability distribution prediction model;

[0058] S4, inputting the power market status data into the trained probability distribution prediction model for processing to obtain the opponent's quotation distribution data;

[0059] S5. Based on the deep deterministic policy gradient algorithm, an Actor-Critic neural network model is constructed; the Actor-Critic neural network model is trained by setting a reinforcement learning toolbox to obtain the trained Actor-Critic neural network model;

[0060] S6. Input the opponent's bid distribution data into the trained Actor-Critic neural network model for processing to obtain the optimal bid quantity and price combination.

[0061] In this embodiment, in step S1, historical transaction data, demand data and price fluctuation data of the electricity market are modeled by a time-generative adversarial network model to generate a simulation data set;

[0062] Specifically, the steps of generating the simulated data set through the temporal generative adversarial network model are as follows:

[0063] S11, pre-processing the collected raw data by setting data cleaning strategy and normalization processing strategy;

[0064] Specifically, the original data is collected, including time series variables such as ambient temperature, humidity, light intensity, market price fluctuations, load forecast results, and adjustable load. The collected original data is recorded at fixed time intervals and data continuity is ensured.

[0065] Process missing data by using interpolation or other statistical methods to fill in missing values; delete outliers and detect and eliminate possible outliers through Z scores or other statistical indicators; use K-means clustering algorithm to cluster and discretize data.

[0066] Normalize the data of each variable to ensure that different variables have similar value ranges. Use the Min-Max normalization formula:

[0067]

[0068] In the formula, x max and x min are the maximum and minimum values ​​of the variable, respectively.

[0069] S12, constructing the time generative adversarial network model, and obtaining the trained time generative adversarial network model through training; generating time series data through the trained time generative adversarial network model;

[0070] Specifically, TimeGAN is a generative adversarial network suitable for time series data generation. By combining the generative adversarial network (GAN) and the self-supervised learning mechanism, it can generate data that conforms to the statistical distribution and retains the temporal dependency.

[0071] TimeGAN consists of four main modules:

[0072] EmbeddingNetwork: maps the input multi-dimensional time series data to a low-dimensional latent space.

[0073] Generator: Generates fake data sequences in the latent space.

[0074] Discriminator: Determines whether the data sequence is forged or real.

[0075] Dynamic Supervisor: Learns to generate temporal dependencies in data.

[0076] Construct the time generative adversarial network model TimeGAN; use the PyTorch framework to implement the TimeGAN model.

[0077] Key parameter settings of the model:

[0078] Hidden layer dimension: 128.

[0079] Time step: 50 steps.

[0080] Activation function: ReLU for embedder and generator, Sigmoid for discriminator.

[0081] Optimizer: Adam optimizer with a learning rate of 0.001.

[0082] The training steps of the TimeGAN model are:

[0083] Use embedder to extract time-dependent features of input data;

[0084] The generator generates fake time series data in the latent space and optimizes the generated results through a dynamic supervisor;

[0085] The discriminator classifies generated data and real data, and the generator improves the generation effect through adversarial training;

[0086] The loss function adopts a multi-task objective:

[0087] Among them, the generator loss is:

[0088]

[0089] In the formula, G(z) represents the forged time series data generated by the generator; D(G(z)) represents the probability that the discriminator determines that the generated data G(z) is real data; Indicates the expected operation; L G represents the average loss over the entire data distribution.

[0090] Discriminator loss:

[0091]

[0092] In the formula, X~p real (X) represents the real data X from the real distribution p real (X) is sampled; D(X) represents the probability that the discriminator determines that the real data X is the real data; z~p z (z) indicates that the input of the generator is from the prior distribution p z Random noise z sampled in (z).

[0093] Dynamic Supervisor Loss:

[0094]

[0095] Where H(X) represents the time-dependent features of the real data X extracted by the embedding network; S(H(X)) represents the feature representation of the dynamic supervisor to further predict the features output by the embedding network; ‖·‖ 2 Represents the quadratic norm, which is used to measure the difference between H(X) and S(H(X)).

[0096] Data Generation:

[0097] After training is completed, the latent variables are input into the generator to generate new time series data.

[0098] The output data has the same time-dependent characteristics as the real data and preserves the statistical relationship between multiple variables.

[0099] S13, performing data denormalization and data verification processing on the time series data to obtain the simulated data set.

[0100] Specifically, the time series data is denormalized to restore the original value range. The formula is as follows:

[0101] x=x′(x max -x min )+x min

[0102] After the data is denormalized, the statistical characteristics of the time series data are compared to ensure that the generated data has similar distribution characteristics such as mean and variance as the real data. Considering the high-dimensional characteristics of historical training data and generated samples, the visual dimensionality reduction T-SNE method is used to compare the sample distribution generated by TimeGAN with the historical training data. The specific results are as follows: Figure 2 shown.

[0103] In this embodiment, in step S2, a probability distribution prediction model is constructed based on a Bayesian network;

[0104] Specifically, a multi-factor driven Bayesian network is established with the bidding decision of building load aggregators as the core. The specific structure is as follows: Figure 3 The network nodes include variables such as environmental factors, market factors, risk preferences, cost structure, load forecasting, and bidding strategies. The nodes in the network are connected by directed edges to describe the causal relationship between different factors.

[0105] Specifically, the environmental factor node is used to represent variables such as ambient temperature, humidity, and light intensity;

[0106] Market factor nodes are used to represent information such as market price fluctuations and demand forecasts;

[0107] Risk preference node, used to indicate the risk tolerance of participants;

[0108] Cost Structure node, used to represent the cost structure of the building load aggregator;

[0109] Load prediction node, used to represent the predicted value of building load;

[0110] The quotation strategy node is used to represent the quotation strategy of the building load aggregator; each node is connected by directed edges to describe the causal relationship between different factors.

[0111] Each node corresponds to a random variable, and its value is calculated based on the conditional probability distribution. The following is an explanation of the calculation method of some key nodes:

[0112] 1. Load Forecasting (LF)

[0113] The calculation formula is:

[0114] LF=f(Temp,Hum,Illum)

[0115] In the formula, f is the load forecasting model, which uses the neural network method and the inputs are temperature, humidity and light intensity. The function f is fitted by training the historical load data.

[0116] 2. Market Price Volatility (MV)

[0117] The calculation formula is:

[0118]

[0119] Where P i is the market price on the i-th day; is the mean of the market price; N is the length of the time series.

[0120] 3. Market Price Forecast (MPF)

[0121] Based on LSTM:

[0122] MPF=f LSTM (MV,history series)

[0123] In the formula, MV is the market price fluctuation, and history series is the historical time series.

[0124] 4. Cost Structure (CS)

[0125] CS = variable cost / fixed cost.

[0126] In this embodiment, in step S3, the probability distribution prediction model is trained by the simulated data set to obtain the trained probability distribution prediction model;

[0127] Specifically, the parameters of the probability distribution prediction model are learned using the generated simulated data set, and the accuracy and stability of the probability distribution prediction model are verified by methods such as cross-validation.

[0128] In this embodiment, in step S4, the power market status data is input into the trained probability distribution prediction model for processing to obtain the opponent's quotation distribution data;

[0129] Specifically, the number and types of competitors participating in the bidding are set according to the market size and actual conditions; the minimum and maximum market demand and demand fluctuation range are set to simulate different market demand scenarios; the bidding strategy range of competitors is set, including the upper and lower limits of the quantity and price combination, to ensure that the generated bidding strategies are reasonable and diverse.

[0130] Specifically, assuming that the demand range of the bidding in each period is, the probability distribution of the opponent's bid is recorded as: [D min ,D max ], the bid volume distribution of the bidding opponents is:

[0131]

[0132] In the formula, q i is the quotation quantity; p i is the quoted price; i is the historical occurrence probability of the corresponding quantity-price combination.

[0133] In this embodiment, in step S5, an Actor-Critic neural network model is constructed based on a deep deterministic policy gradient algorithm; the Actor-Critic neural network model is trained by setting a reinforcement learning toolbox to obtain the trained Actor-Critic neural network model;

[0134] Specifically, the Actor-Critic neural network model includes an Actor neural network and a Critic neural network; the Actor neural network is used to receive the state input of the current environment and generate corresponding action outputs; specifically, the network first linearly transforms the input state information through a set of parameterized weights and biases, and then uses the ReLU activation function to perform nonlinear processing on the transformation results to extract key features. Then, these features are further passed to the output layer, linearly transformed through another set of weights and biases, and finally the Sigmoid activation function is used to compress the results to a predetermined range to generate the final action output. The Critic neural network is used to receive the joint input of state and action, evaluate the quality of the generated action and output the corresponding Q value. Specifically, the state information is first linearly transformed through a set of weights and biases and processed by the ReLU activation function to generate state features; the action information is linearly transformed through another set of independent weights and biases, and is also processed by the ReLU activation function to generate action features. Subsequently, the state features and action features are combined into a feature vector through a concatenation operation and input to the next layer. A linear transformation is completed through a set of parameterized weights and biases, and finally the Q' value is output, which represents the expected benefit value of taking the action in the current state. In this embodiment, the weights and biases of the Actor network and the Critic network are both trainable parameters, which are adjusted through the optimization method in deep reinforcement learning to ensure the performance and stability of the network in complex environments.

[0135] Define the market state space as a four-dimensional vector including current demand, future demand, liquidated quantity and marginal price;

[0136] The state vector in this embodiment is defined as:

[0137] s t =[D t ,D t-1 ,Q cleared ,P margin ]

[0138] Where D t Indicates the demand in the current period; D t-1 Indicates the demand in the previous period; Q cleared Indicates the amount of liquidation by our aggregator in the previous period; P marginRepresents the marginal price in the previous period.

[0139] The action space is defined as the quantity-price combination for each time period, with a total of 18-dimensional action vectors;

[0140] In this embodiment, assuming that there are M pairs of quantity-price combinations in each time period, the action vector is defined as:

[0141] a t =[q 1,t ,p 1,t ,...,q M,t ,p M,t ],

[0142] In the formula, p i,t and q i,t Represent the quantity and price of the i-th quantity-price combination respectively.

[0143] The action value is limited to [0,1] by the Sigmoid function and further scaled to the actual range:

[0144]

[0145] In the formula, q i,t is the actual bidding quantity; is the normalized bid quantity; Q max is the maximum bidding quantity; p i,t is the actual bid price; is the normalized bid price; P max The highest bid price.

[0146] Build an Actor-Critic neural network architecture based on the Deep Deterministic Policy Gradient (DDPG) algorithm and use GPU to accelerate training;

[0147] Specifically, the training steps are:

[0148] S51, inputting the initial demand data and the opponent's quotation distribution data into the Actor-Critic neural network model;

[0149] S52, the Actor network in the Actor-Critic neural network model processes the initial demand data and the opponent's quotation distribution data to generate an action output;

[0150] S53, the Critic network in the Actor-Critic neural network model performs quality assessment on the action output and outputs a Q value;

[0151] S54, the Actor network updates the parameters of the Actor-Critic neural network model based on the policy gradient according to the Q value, and obtains a reward through environmental function calculation;

[0152] Specifically, in each decision step, through N sim Monte Carlo simulation to calculate single-step profit:

[0153]

[0154] In the formula, C unit is the unit cost; Q cleared,i ,P margin,i are the liquidation quantity and marginal price respectively.

[0155] Define the conditional value at risk (CVaR) at the α confidence level as:

[0156]

[0157] In the formula, α is the confidence level; (i) Represents the sorting result of profits.

[0158] Combining average profit with CVaR to construct a reward function:

[0159]

[0160] S55. When the reward function converges, stop training and obtain the trained Actor-Critic neural network model.

[0161] Specifically, observe the training status through the MATLAB reinforcement learning training window, and end the training when the reward function converges, such as Figure 4 shown.

[0162] After training is complete, the agent model is saved in .mat format. After loading this file, the parameters learned during training can be restored. Before calling the model, you need to define an environment (env) compatible with the model, which includes the state space, action space, and reward function. By loading the state and action space parameters defined during training.

[0163] In this embodiment, in step S6, the opponent's bid distribution data is input into the trained Actor-Critic neural network model for processing to obtain the optimal bid quantity and price combination.

[0164] Specifically, the actual market status is input to obtain the original action, and after mapping, the optimal bidding quantity and price combination can be obtained.

[0165] In summary, the present invention has the following advantages: the present invention models the historical transaction data, demand data and price fluctuation data of the power market through the time generative adversarial network model to generate a simulation data set; based on the Bayesian network, a probability distribution prediction model is constructed; the probability distribution prediction model is trained through the simulation data set to obtain the trained probability distribution prediction model; the power market state data is input into the trained probability distribution prediction model for processing to obtain the opponent's quotation distribution data; based on the deep deterministic policy gradient algorithm, an Actor-Critic neural network model is constructed; the Actor-Critic neural network model is trained by setting a reinforcement learning toolbox to obtain the trained Actor-Critic neural network model; the opponent's quotation distribution data is input into the trained Actor-Critic neural network model for processing to obtain the optimal bidding quantity and price combination. The present invention can comprehensively consider the power market environment of various factors and provide a method for the optimal bidding quantity and price combination for building load aggregators. The present invention generates market data based on the time generative adversarial network (TimeGAN), constructs the power market Bayesian network, learns the probability distribution of opponent behavior, and optimizes the quotation and quotation decision through deep reinforcement learning (DDPG). The intelligent agent combines market demand, price fluctuations and other factors to generate the optimal bidding strategy to balance benefits and risks and maximize long-term profits. The system improves the accuracy and adaptability of power market bidding decisions through simulating market environments and strengthening learning training, and promotes the intelligent development of the market.

[0166] It should be noted that the method of the embodiment of the present disclosure can be performed by a single device, such as a computer or a server. The method of the present embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present disclosure, and the multiple devices will interact with each other to complete the described method.

[0167] It should be noted that the above describes some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0168] Example 2

[0169] See also Figure 5Embodiment 2 of the present invention further provides a decision-making device for a building load aggregator to participate in a power market bidding, comprising:

[0170] A simulation data set generation module 001 is used to model the historical transaction data, demand data and price fluctuation data of the power market through a time-generated adversarial network model to generate a simulation data set;

[0171] The probability distribution prediction model construction module 002 is used to construct a probability distribution prediction model based on a Bayesian network;

[0172] The probability distribution prediction model training module 003 is used to train the probability distribution prediction model through the simulated data set to obtain the trained probability distribution prediction model;

[0173] The probability distribution prediction model prediction processing module 004 is used to input the power market status data into the trained probability distribution prediction model for processing to obtain the opponent's quotation distribution data;

[0174] Actor-Critic neural network model construction and training module 005, used to construct an Actor-Critic neural network model based on a deep deterministic policy gradient algorithm; train the Actor-Critic neural network model by setting a reinforcement learning toolbox to obtain the trained Actor-Critic neural network model;

[0175] The Actor-Critic neural network model processing module 006 is used to input the opponent's bid distribution data into the trained Actor-Critic neural network model for processing to obtain the optimal bid quantity and price combination.

[0176] In this embodiment, in the simulation data set generation module 001, the submodules for generating the simulation data set include:

[0177] The raw data preprocessing submodule 011 is used to preprocess the collected raw data by setting a data cleaning strategy and a normalization processing strategy;

[0178] The temporal generative adversarial network model construction and processing submodule 012 is used to construct the temporal generative adversarial network model and obtain the trained temporal generative adversarial network model through training; and generate time series data through the trained temporal generative adversarial network model;

[0179] The simulation data set generation submodule 013 is used to perform data denormalization and data verification processing on the time series data to obtain the simulation data set.

[0180] In this embodiment, in the probability distribution prediction model construction module 002, the nodes in the probability distribution prediction model include: environmental factor nodes, market factor nodes, risk preference nodes, cost structure nodes, load forecast nodes and quotation strategy nodes; each node is associated with each other through directed edges.

[0181] In this embodiment, in the Actor-Critic neural network model construction and training module 005, the Actor-Critic neural network model includes an Actor neural network and a Critic neural network; the Actor neural network is used to receive the state input of the current environment and generate corresponding action output; the Critic neural network is used to receive the joint input of the state and action, evaluate the quality of the generated action and output the corresponding Q value.

[0182] In this embodiment, in the Actor-Critic neural network model construction and training module 005, the Actor-Critic neural network model is trained using a GPU through the DDPG deep reinforcement learning toolbox in MATLAB; the submodules for training the Actor-Critic neural network model include:

[0183] The data input submodule 051 is used to input the initial demand data and the opponent's quotation distribution data into the Actor-Critic neural network model;

[0184] Action output generation submodule 052, used for the Actor network in the Actor-Critic neural network model to process the initial demand data and the opponent's quotation distribution data to generate action output;

[0185] An action output quality assessment submodule 053 is used for the Critic network in the Actor-Critic neural network model to perform quality assessment on the action output and output a Q value;

[0186] A parameter updating submodule 054 is used for the Actor network to update the parameters of the Actor-Critic neural network model based on the policy gradient according to the Q value, and obtain a reward through environmental function calculation;

[0187] The trained Actor-Critic neural network model acquisition submodule 055 is used to stop training when the reward function converges and obtain the trained Actor-Critic neural network model.

[0188] It should be noted that the information interaction, execution process and other contents between the modules of the above-mentioned system are based on the same concept as the method embodiment in Example 1 of the present application, and the technical effects they bring are the same as those of the method embodiment of the present application. For specific contents, please refer to the description in the method embodiment shown above in the present application, and will not be repeated here.

[0189] Example 3

[0190] Embodiment 3 of the present invention provides a non-transitory computer-readable storage medium, in which a program code for a method for a building load aggregator to participate in electricity market bidding decision-making is stored. The program code includes instructions for executing embodiment 1 or any possible implementation method of a method for a building load aggregator to participate in electricity market bidding decision-making.

[0191] The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0192] Example 4

[0193] Embodiment 4 of the present invention provides an electronic device, including: a memory and a processor;

[0194] The processor and the memory communicate with each other via a bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute a method for building load aggregators to participate in electricity market bidding decision-making in embodiment 1 or any possible implementation thereof.

[0195] Specifically, the processor can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor can be a general-purpose processor implemented by reading software codes stored in a memory. The memory can be integrated in the processor or can be located outside the processor and exist independently.

[0196] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable systems. The computer instructions can be stored in a computer-readable storage medium, or transmitted from a computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center.

[0197] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing system, they can be concentrated on a single computing system, or distributed on a network composed of multiple computing systems, and optionally, they can be implemented by a program code executable by a computing system, so that they can be stored in a storage system and executed by the computing system, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0198] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A decision-making method for building load aggregators to participate in power market bidding, characterized in that: include: The historical transaction data, demand data and price fluctuation data of the electricity market are modeled through the time-generated adversarial network model to generate a simulated data set; Based on Bayesian network, construct probability distribution prediction model; Training the probability distribution prediction model using the simulated data set to obtain a trained probability distribution prediction model; Inputting the power market status data into the trained probability distribution prediction model for processing to obtain the opponent's quotation distribution data; Based on the deep deterministic policy gradient algorithm, an Actor-Critic neural network model is constructed; the Actor-Critic neural network model is trained by setting a reinforcement learning toolbox to obtain the trained Actor-Critic neural network model; The opponent's bid distribution data is input into the trained Actor-Critic neural network model for processing to obtain the optimal bid quantity and price combination.

2. A decision-making method for building load aggregators to participate in power market bidding according to claim 1, characterized in that: The steps of generating the simulated data set by the temporal generative adversarial network model are as follows: Pre-process the collected raw data by setting data cleaning strategy and normalization processing strategy; Constructing the temporal generative adversarial network model, and obtaining the trained temporal generative adversarial network model through training; generating time series data through the trained temporal generative adversarial network model; The time series data is subjected to data denormalization processing and data verification processing to obtain the simulated data set.

3. A decision-making method for building load aggregators to participate in power market bidding according to claim 2, characterized in that: The nodes in the probability distribution prediction model include: environmental factor nodes, market factor nodes, risk preference nodes, cost structure nodes, load forecast nodes and quotation strategy nodes; the nodes are associated with each other through directed edges.

4. A decision-making method for building load aggregators to participate in power market bidding according to claim 3, characterized in that: The Actor-Critic neural network model includes an Actor neural network and a Critic neural network; the Actor neural network is used to receive the state input of the current environment and generate corresponding action output; the Critic neural network is used to receive the joint input of the state and action, evaluate the quality of the generated action and output the corresponding Q value.

5. A decision-making method for building load aggregators to participate in power market bidding according to claim 4, characterized in that: In the process of training the Actor-Critic neural network model by using the set reinforcement learning toolbox, the Actor-Critic neural network model is trained by using the GPU through the DDPG deep reinforcement learning toolbox in MATLAB; the training steps are: Inputting the initial demand data and the counterparty quotation distribution data into the Actor-Critic neural network model; The Actor network in the Actor-Critic neural network model processes the initial demand data and the opponent's quotation distribution data to generate an action output; The Critic network in the Actor-Critic neural network model performs quality assessment on the action output and outputs a Q value; The Actor network updates the parameters of the Actor-Critic neural network model based on the policy gradient according to the Q value, and obtains a reward through environmental function calculation; When the reward function converges, the training is stopped to obtain the trained Actor-Critic neural network model.

6. A decision-making device for building load aggregators to participate in power market bidding, adopting a decision-making method for building load aggregators to participate in power market bidding as described in any one of claims 1 to 5, characterized in that: include: A simulation data set generation module is used to model the historical transaction data, demand data and price fluctuation data of the power market through a time-generated adversarial network model to generate a simulation data set; Probability distribution prediction model building module, used to build a probability distribution prediction model based on Bayesian network; A probability distribution prediction model training module, used to train the probability distribution prediction model through the simulated data set to obtain the trained probability distribution prediction model; A probability distribution prediction model prediction processing module is used to input the power market status data into the trained probability distribution prediction model for processing to obtain the opponent's quotation distribution data; Actor-Critic neural network model construction and training module, used to construct an Actor-Critic neural network model based on a deep deterministic policy gradient algorithm; train the Actor-Critic neural network model by setting a reinforcement learning toolbox to obtain the trained Actor-Critic neural network model; The Actor-Critic neural network model processing module is used to input the opponent's quotation distribution data into the trained Actor-Critic neural network model for processing to obtain the optimal bidding quantity and price combination.

7. A decision-making device for building load aggregators to participate in power market bidding according to claim 6, characterized in that: In the simulation data set generation module, the submodule for generating the simulation data set includes: The raw data preprocessing submodule is used to preprocess the collected raw data by setting data cleaning strategy and normalization processing strategy; A temporal generative adversarial network model construction and processing submodule is used to construct the temporal generative adversarial network model and obtain the trained temporal generative adversarial network model through training; and generate time series data through the trained temporal generative adversarial network model; The simulation data set generation submodule is used to perform data denormalization and data verification processing on the time series data to obtain the simulation data set.

8. The decision-making device for building load aggregators to participate in power market bidding according to claim 7, characterized in that: In the probability distribution prediction model construction module, the nodes in the probability distribution prediction model include: environmental factor nodes, market factor nodes, risk preference nodes, cost structure nodes, load forecast nodes and quotation strategy nodes; the nodes are associated with each other through directed edges.

9. A decision-making device for building load aggregators to participate in power market bidding according to claim 8, characterized in that: In the Actor-Critic neural network model construction and training module, the Actor-Critic neural network model includes an Actor neural network and a Critic neural network; the Actor neural network is used to receive the state input of the current environment and generate corresponding action output; the Critic neural network is used to receive the joint input of the state and action, evaluate the quality of the generated action and output the corresponding Q value.

10. A decision-making device for building load aggregators to participate in power market bidding according to claim 9, characterized in that: In the Actor-Critic neural network model construction and training module, the Actor-Critic neural network model is trained using a GPU through the DDPG deep reinforcement learning toolbox in MATLAB; The submodules for training the Actor-Critic neural network model include: A data input submodule, used to input the initial demand data and the opponent's quotation distribution data into the Actor-Critic neural network model; An action output generation submodule, used for the Actor network in the Actor-Critic neural network model of the Actor network to process the initial demand data and the opponent's quotation distribution data to generate an action output; An action output quality assessment submodule, used for the Critic network in the Actor-Critic neural network model to perform quality assessment on the action output and output a Q value; A parameter updating submodule, used for the Actor network to update the parameters of the Actor-Critic neural network model based on the policy gradient according to the Q value, and obtain rewards through environmental function calculation; The trained Actor-Critic neural network model acquisition submodule is used to stop training when the reward function converges and obtain the trained Actor-Critic neural network model.

Images