Bidding method and device for participating in electric power spot market through energy storage quantity reporting and quotation

Abstract

The invention relates to a bidding method and device for participation of energy storage quantity reporting and quoting in an electric power spot market, and the method comprises the steps: constructing a state space model and a reward function model of an energy storage main body in the electric power market, and constructing a price response type bidding neural network of an energy storage intelligent body in an electric power market price environment; training a bidding neural network based on a PPO algorithm; based on a price response neural network, an association relationship from the electricity price to the power of the energy storage unit is sampled to obtain a corresponding supply curve, and the supply curve is monotonized to meet the monotonically increasing requirement of the market on a bidding object; the monotonized supply curve is discretized into price-power pairs to generate an offer bid that meets market requirements. Therefore, the problems that in the prior art, a high-dimensional report and quotation format in the electricity market cannot be effectively utilized, the potential of the report and quotation format is difficult to give full play, and the degree of freedom of bidding is prone to being damaged are solved.

Description

Technical Field

[0001] The present application relates to the technical field of bidding in the electricity spot market, and in particular to a bidding method and device for bidding for energy storage quantity and quotation in the electricity spot market. Background Art

[0002] In recent years, the volatility and randomness of renewable energy generation have led to a more random and uncertain electricity market price. The increase in market price uncertainty has posed greater challenges to the bidding of market players. In order to achieve high returns under high market uncertainty, energy storage power generation companies must wisely formulate their market bidding strategies to ensure higher market profits.

[0003] In order to cope with the rising uncertainty in the electricity market, methods based on deep reinforcement learning have been widely adopted to solve the market uncertainty problem in the real world. Specifically, deep reinforcement learning methods directly learn bidding strategies by interacting with the simulated real bidding process to manage price uncertainty and achieve good bidding results. At the same time, in most electricity markets, market players bid by quoting quantities; the subject of a quantity quotation is composed of several price-power pairs arranged in a monotonically increasing order. In the usual electricity market, the dimensions of price and power are higher than twenty dimensions. Such a high-dimensional bidding space poses a challenge to the effectiveness of deep reinforcement learning algorithms.

[0004] Methods based on deep reinforcement learning can cope with real-world uncertainties by learning from real-world market data. However, current deep reinforcement learning methods cannot effectively utilize high-dimensional bidding formats in the power market. Instead, they can only simplify the bidding space and bid in low-dimensional bidding formats, such as single-value price bidding, single-value power bidding, and a pair of price-power bidding. These bidding formats only utilize less than five dimensions in the bidding format and do not fully utilize the potential of the bidding format. Alternatively, some deep reinforcement learning bidding methods use high-dimensional bidding formats in an indirect way. However, existing technologies use specific simplifications related to subject behavior to convert high-dimensional bidding spaces into low-dimensional spaces, which inevitably damages the freedom of bidding.

[0005] In summary, the existing technology cannot effectively utilize the high-dimensional quotation format in the electricity market, it is difficult to fully tap the potential of the quotation format, and it is easy to damage the freedom of bidding, which needs to be solved urgently. Summary of the invention

[0006] The present application provides a bidding method and device for energy storage quantity quotation to participate in the electricity spot market, so as to solve the problems that the existing technology cannot effectively utilize the high-dimensional quantity quotation format in the electricity market, it is difficult to fully realize the potential of the quantity quotation format, and it is easy to damage the freedom of bidding.

[0007] The first aspect of the present application provides a bidding method for energy storage to bid for participating in the electricity spot market, which is applied to the offline training stage and includes the following steps: constructing a state space model and a reward function model of a target energy storage entity operating in a target electricity market, constructing a price-responsive bidding neural network for an energy storage intelligent entity corresponding to the target energy storage entity under the target electricity market price environment, and obtaining environmental information and market prices of the target energy storage entity, and inputting the environmental information and the market price into the bidding neural network to output the corresponding self-scheduling power; training the bidding neural network based on the state space model, the reward function model and the preset PPO algorithm, and in combination with the environmental information, the market price and the self-scheduling power to obtain a target energy storage bidding neural network model.

[0008] Optionally, in one embodiment of the present application, the state space model and reward function model of the target energy storage entity operating in the target electricity market are constructed, including: obtaining the energy market clearing price in the target electricity market, and determining the market bidding income according to the energy market clearing price; obtaining the discharge power and energy storage depreciation cost parameters corresponding to the target energy storage entity, and calculating the energy storage depreciation cost through the discharge power and the energy storage depreciation cost parameters; determining the SoC out-of-bounds penalty corresponding to the target energy storage entity according to the state of charge out-of-bounds penalty constant and the maximum state of charge corresponding to the target energy storage entity; and constructing a reward function model based on the market bidding income, the energy storage depreciation cost and the SoC out-of-bounds penalty.

[0009] Optionally, in one embodiment of the present application, the price-responsive bidding neural network of the energy storage agent corresponding to the target energy storage subject in the target electricity market price environment is constructed, and the environmental information and market price of the target energy storage subject are obtained, and the environmental information and the market price are input into the bidding neural network to output the corresponding self-dispatching power, including: based on a preset multi-layer perceptron network, constructing a strategy function network and a value function network for the bidding of the energy storage agent, and using the strategy function network and the value function network to establish the bidding neural network; obtaining the bidding time of the energy storage agent, and performing Perform trigonometric function coding to obtain a corresponding coding result; collect the market price of the energy storage intelligent entity in the target time period, and perform discrete Fourier transform on the market price to obtain a corresponding discrete Fourier transform result, and extract the angle and amplitude values ​​of the first three dimensions in the discrete Fourier transform result; obtain the charge state information and the market price corresponding to the energy storage intelligent entity, and determine the environmental information according to the charge state information, the coding result, the angle and the amplitude value; input the environmental information and the market price into the bidding neural network to output the self-dispatching power of the target energy storage entity in the target power market.

[0010] The second aspect of the present application provides a bidding method for energy storage quantity quotation to participate in the electricity spot market, which is applied to the online deployment stage, and includes the following steps: obtaining multiple market electricity prices corresponding to preset energy storage entities, and inputting each of the multiple market electricity prices into a pre-constructed target energy storage bidding neural network model in turn to output the energy storage unit power corresponding to each market electricity price, wherein the target energy storage bidding neural network model is obtained by training a preset bidding neural network with a pre-state space model, a reward function model and a PPO algorithm; uniformly sampling the energy storage unit power output by the target energy storage bidding neural network model and the market electricity price corresponding to the energy storage unit power to obtain corresponding multiple associated sampling results, and constructing a unit supply curve based on the multiple associated sampling results; performing monotonicity and discretization operations on the unit supply curve to generate a quantity quotation bid that meets preset market requirements.

[0011] Optionally, in one embodiment of the present application, the monotonicization and discretization operations are performed on the unit supply curve to generate a target quantity quotation bid that meets preset market requirements, including: performing a monotonicization operation on the unit supply curve to generate a monotonic unit supply curve that meets the monotonic increasing requirements of the preset market bidding target, and discretizing the monotonic unit supply curve to obtain a plurality of quantity quotation bid pairs; based on a preset initial bidding pair, performing equally spaced sampling among the plurality of quantity quotation bid pairs to obtain a plurality of bidding pair sampling data; iteratively calculating the change ratio corresponding to each bidding pair sampling data in the plurality of bidding pair sampling data through a preset iterative algorithm until the change ratio corresponding to each bidding pair sampling data is less than a preset threshold value, so as to generate the target quantity quotation bid.

[0012] Optionally, in one embodiment of the present application, the mathematical expression of the preset iterative algorithm is:

[0013]

[0014] Among them, λ i represents the iteratively generated bidding price; p i represents the iteratively generated bidding power; λ k represents the market price in the sample data of the plurality of bid pairs; k The power in the sampled data of the plurality of bidding pairs is represented.

[0015] According to a third aspect of the present application, there is provided a bidding device for energy storage to bid for participating in the electricity spot market, which is applied to an offline training stage and includes: a first modeling module for constructing a state space model and a reward function model of a target energy storage entity operating in a target electricity market; a second modeling module for constructing a price-responsive bidding neural network of an energy storage intelligent entity corresponding to the target energy storage entity under the target electricity market price environment, and obtaining environmental information and market prices of the target energy storage entity, and inputting the environmental information and the market prices into the bidding neural network to output the corresponding self-scheduling power; a training module for training the bidding neural network based on the state space model, the reward function model and a preset PPO algorithm, and in combination with the environmental information, the market price and the self-scheduling power to obtain a target energy storage bidding neural network model.

[0016] Optionally, in one embodiment of the present application, the first modeling module includes: a first acquisition unit, used to obtain the energy market clearing price in the target power market, and determine the market bidding income based on the energy market clearing price; a second acquisition unit, used to obtain the discharge power and energy storage depreciation cost parameters corresponding to the target energy storage entity, and calculate the energy storage depreciation cost through the discharge power and the energy storage depreciation cost parameters; a first determination unit, used to determine the SoC out-of-bounds penalty corresponding to the target energy storage entity based on the state of charge out-of-bounds penalty constant and maximum state of charge corresponding to the target energy storage entity; a construction unit, used to construct a reward function model based on the market bidding income, the energy storage depreciation cost and the SoC out-of-bounds penalty.

[0017] Optionally, in one embodiment of the present application, the second modeling module includes: an establishment unit, which is used to construct a strategy function network and a value function network for bidding the energy storage agent based on a preset multilayer perceptron network, and use the strategy function network and the value function network to establish the bidding neural network; an encoding unit, which is used to obtain the bidding time of the energy storage agent, and perform trigonometric function encoding on the bidding time to obtain a corresponding encoding result; a transformation unit, which is used to collect the market price of the energy storage agent in the target time period, and perform discrete Fourier transform on the market price to obtain a corresponding discrete Fourier transform result, and extract the angle and amplitude of the first three dimensions in the discrete Fourier transform result; a second determination unit, which is used to obtain the state of charge information and the market price corresponding to the energy storage agent, and determine the environmental information according to the state of charge information, the encoding result, the angle and the amplitude; an input unit, which is used to input the environmental information and the market price into the bidding neural network to output the self-dispatching power of the target energy storage subject in the target power market.

[0018] The fourth aspect of the present application provides a bidding device for energy storage quantity quotation and participation in the electricity spot market, which is applied to the online deployment stage and includes: a deployment module, which is used to obtain multiple market electricity prices corresponding to a preset energy storage entity, and input each of the multiple market electricity prices into a pre-constructed target energy storage bidding neural network model in turn to output the energy storage unit power corresponding to each market electricity price, wherein the target energy storage bidding neural network model is obtained by training a preset bidding neural network with a pre-state space model, a reward function model and a PPO algorithm; a sampling module, which is used to uniformly sample the energy storage unit power output by the target energy storage bidding neural network model and the market electricity price corresponding to the energy storage unit power to obtain corresponding multiple associated sampling results, and construct a unit supply curve according to the multiple associated sampling results; a generation module, which is used to perform monotonicity and discretization operations on the unit supply curve to generate a quantity quotation bid that meets preset market requirements.

[0019] Optionally, in one embodiment of the present application, the generation module includes: a monotonicization unit, which is used to perform a monotonicization operation on the unit supply curve to generate a monotonic unit supply curve that meets the monotonic increasing requirements of a preset market bidding target, and discretize the monotonic unit supply curve to obtain multiple quantity quotation and bid pairs; a processing unit, which is used to perform equally spaced sampling in the multiple quantity quotation and bid pairs based on a preset initial bid pair to obtain multiple bid pair sampling data; an iteration unit, which is used to iteratively calculate the change ratio corresponding to each bid pair sampling data in the multiple bid pair sampling data through a preset iterative algorithm until the change ratio corresponding to each bid pair sampling data is less than a preset threshold value, so as to generate the target quantity quotation and bid.

[0020] Optionally, in one embodiment of the present application, the mathematical expression of the preset iterative algorithm is:

[0021]

[0022] Among them, λ i represents the iteratively generated bidding price; p i represents the iteratively generated bidding power; λ k represents the market price in the sample data of the plurality of bid pairs; k The power in the sampled data of the plurality of bidding pairs is represented.

[0023] The fifth aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the bidding method for energy storage quantity quotation to participate in the electricity spot market as described in the above embodiment.

[0024] A sixth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-mentioned method for bidding for energy storage quantity and quotation in the electricity spot market.

[0025] Therefore, the embodiments of the present application have the following beneficial effects:

[0026] The embodiments of the present application can construct a state space model and a reward function model of the energy storage subject operating in the power market, and construct a price-responsive bidding neural network for the energy storage intelligent body in the power market price environment; train the bidding neural network based on the PPO algorithm; based on the price-responsive neural network, sample the correlation from the electricity price to the power of the energy storage unit to obtain the corresponding supply curve, and monotonize the supply curve to meet the market's requirements for monotonically increasing bidding targets; discretize the monotonized supply curve into price-power pairs to generate a bid quotation that meets market requirements, so as to release the bidding potential of the bid quotation format more accurately and effectively, and give full play to the advantages of deep reinforcement learning methods in dealing with price uncertainty. Thus, the problems that the existing technology cannot effectively utilize the high-dimensional bid quotation format in the power market, it is difficult to give full play to the potential of the bid quotation format, and it is easy to damage the freedom of bidding are solved.

[0027] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0029] Figure 1 A flowchart of a bidding method for energy storage quotation and participation in the electricity spot market in an offline training phase provided according to an embodiment of the present application;

[0030] Figure 2 A flowchart of a bidding method for energy storage quotation and participation in the electricity spot market in an online deployment phase provided according to an embodiment of the present application;

[0031] Figure 3 This is an example diagram of a bidding device for energy storage quotation and participation in the electricity spot market in an offline training phase according to an embodiment of the present application;

[0032] Figure 4 This is an example diagram of a bidding device for energy storage quotation and participation in the electricity spot market in the online deployment phase according to an embodiment of the present application;

[0033] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0034] Among them, 10 is a bidding device for energy storage quantity quotation participating in the electricity spot market during the offline training stage, 20 is a bidding device for energy storage quantity quotation participating in the electricity spot market during the online deployment stage; 101 is a first modeling module, 102 is a second modeling module, 103 is a training module; 201 is a deployment module, 202 is a sampling module, 203 is a generation module; 501 is a memory, 502 is a processor, and 503 is a communication interface. DETAILED DESCRIPTION

[0035] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0036] The following is a description of the bidding method and device for energy storage quotation and quotation in the electricity spot market according to the embodiment of the present application with reference to the accompanying drawings. In the method, a state space model and a reward function model of the energy storage subject operating in the electricity market are constructed, and a price-responsive bidding neural network of the energy storage intelligent body in the electricity market price environment is constructed; the bidding neural network is trained based on the PPO algorithm; based on the price-responsive neural network, the correlation relationship from the electricity price to the power of the energy storage unit is sampled to obtain the corresponding supply curve, and the supply curve is monotonized to meet the market's monotonically increasing requirements for the bidding target; the monotonized supply curve is discretized into price-power pairs to generate a quotation and quotation bidding that meets market requirements, so that the bidding potential of the quotation and quotation format can be released more accurately and effectively, and the advantages of deep reinforcement learning methods in dealing with price uncertainty are fully utilized. Thus, the problems that the prior art cannot effectively utilize the high-dimensional quotation and quotation format in the electricity market, it is difficult to fully utilize the potential of the quotation and quotation format, and it is easy to damage the freedom of bidding are solved.

[0037] In order to facilitate those skilled in the art to understand the execution logic of the bidding method for energy storage quotation to participate in the electricity spot market of the present application, the following describes the parameters and their interpretations involved in the present application, and briefly summarizes the execution logic of the present application:

[0038] 1. Parameters and their meanings:

[0039] Time step (unit is the duration of a single bid): t

[0040] Time step: τ

[0041] Price and Price Bid: λ

[0042] Power and Power Bid:p

[0043] Minimum price: λ min

[0044] Maximum price: λ max

[0045] Minimum power: p min

[0046] Maximum power: p max

[0047] State of Charge of Energy Storage: SoC

[0048] Maximum state of charge: SoC max

[0049] Minimum state of charge: SoC min

[0050] Charging efficiency: η c

[0051] Discharge efficiency: η d

[0052] Bidding objective (reward function): r t

[0053] Cost parameter for energy storage depreciation: λ dep

[0054] The cost of energy storage depreciation:

[0055] Charge state out-of-bounds penalty constant: P

[0056] Energy storage charging status violation penalties: Reinforcement learning discount factor: γ

[0057] 2. Execution logic:

[0058] This application generates high-dimensional quotation bids in two stages:

[0059] 1. Offline training phase:

[0060] In this stage, a price-responsive bidding strategy based on reinforcement learning for energy storage in the power market price environment is used to train a neural network that can respond strategically to power market prices, thereby achieving effective training of the bidding algorithm.

[0061] 2. Online deployment stage:

[0062] This stage uses the neural network trained in the offline training phase to generate strategic energy storage quotation bidding targets, so that the neural network results trained in the offline training phase can be directly deployed in the power market.

[0063] Specifically, Figure 1 A flowchart of a bidding method for energy storage quotation in an offline training phase to participate in the electricity spot market provided in an embodiment of the present application.

[0064] like Figure 1 As shown, the bidding method for energy storage quotation to participate in the electricity spot market includes the following steps:

[0065] In step S101, a state space model and a reward function model of a target energy storage entity operating in a target power market are constructed.

[0066] The embodiments of the present application can first construct a state space model and reward function model of the energy storage entity operating in the electricity market to describe its state transition function and reward function, and combine market revenue, depreciation cost and charge state out-of-bounds penalty to construct the operation problem of the energy storage system in the market.

[0067] Optionally, in one embodiment of the present application, a state space model and a reward function model of the target energy storage entity operating in the target electricity market are constructed, including: obtaining the energy market clearing price in the target electricity market, and determining the market bidding income based on the energy market clearing price; obtaining the discharge power and energy storage depreciation cost parameters corresponding to the target energy storage entity, and calculating the energy storage depreciation cost through the discharge power and energy storage depreciation cost parameters; determining the SoC out-of-bounds penalty corresponding to the target energy storage entity based on the state of charge out-of-bounds penalty constant and the maximum state of charge corresponding to the target energy storage entity; and constructing a reward function model based on the market bidding income, energy storage depreciation cost and SoC out-of-bounds penalty.

[0068] As an achievable method, the embodiment of the present application proposes an energy storage model based on lithium-ion batteries. The real-time energy market is one of the most profitable markets for energy storage to participate in energy arbitrage, and the proposed energy storage model takes into account its state transition and its bidding target.

[0069] In an embodiment of the present application, the internal state of the energy storage can be obtained according to the state of charge (SoC) model, as shown in the following formula:

[0070]

[0071] Secondly, the bidding target of energy storage (i.e., the reward function model) includes three parts: market bidding income, energy storage depreciation cost, and penalty for SoC out-of-bounds, as shown in the following formula:

[0072]

[0073] Among them, λ t The energy market clearing price; the embodiment of the present application considers a storage depreciation model in which the cost is proportional to the discharge power and time. The depreciation cost model of the energy storage It can be expressed as:

[0074]

[0075] In addition, during the reinforcement learning training process, the behavior of the reinforcement learning algorithm may cause the SoC to exceed the upper and lower limits of the energy storage energy. Therefore, the embodiment of the present application may add additional penalties during the reinforcement learning training process to avoid SoC violations. The SoC violation penalty (i.e., the SoC out-of-bounds penalty) can be expressed as:

[0076]

[0077] Therefore, the embodiments of the present application provide reliable technical and data support for the subsequent training of the energy storage bidding neural network by constructing a state space model and a reward function model.

[0078] In step S102, a price-responsive bidding neural network is constructed for the energy storage intelligent entity corresponding to the target energy storage entity under the target electricity market price environment, and the environmental information and market price of the target energy storage entity are obtained, and the environmental information and market price are input into the bidding neural network to output the corresponding self-dispatching power.

[0079] In step S103, based on the state space model, the reward function model and the preset PPO algorithm, and in combination with environmental information, market price and self-dispatching power, a bidding neural network is trained to obtain a target energy storage bidding neural network model.

[0080] Furthermore, the embodiments of the present application also need to adopt a multi-layer perceptron neural network and input time information, market history and current state of charge to output power scheduling decisions, thereby establishing a price response bidding neural network for energy storage intelligent entities in the electricity market price environment.

[0081] Afterwards, the embodiments of the present application can utilize the state space model and reward function model constructed above, and combine the preset PPO (Proximal Policy Optimization) algorithm, as well as environmental information, market prices and self-scheduling power to train the bidding neural network, thereby obtaining the target energy storage bidding neural network model.

[0082] It should be noted that in the embodiments of the present application, the PPO algorithm combines the proposed state space model, reward function model and bidding neural network into a Markov decision process, and calculates the policy gradient of the neural network optimization through iterative interaction between the neural network and the state space model and reward function model, thereby realizing the learning of the bidding strategy.

[0083] Specifically, first, in N training rounds of length T, the market state is sampled at each time step t and the market clearing price λ t ; Secondly, the embodiment of the present application can combine the market status and price into the current status Again, the embodiment of the present application can sample the power p from the output distribution of the price response neural network t ~π θ (s t ); The embodiment of the present application calculates the reward r through the reward function t , and move to the next time step, storing the time step history in the round buffer D. After all time steps are completed, the round buffer is used to update the function estimate; based on the advantage of the current calculation time step Update the policy π by maximizing the PPO objective function θ ; Finally, the embodiment of the present application clears the round buffer D to enter the next round of training.

[0084] Optionally, in one embodiment of the present application, a price-responsive bidding neural network is constructed for the energy storage agent corresponding to the target energy storage subject in the target electricity market price environment, and the environmental information and market price of the target energy storage subject are obtained, and the environmental information and market price are input into the bidding neural network to output the corresponding self-dispatching power, including: based on a preset multi-layer perceptron network, a strategy function network and a value function network for the bidding of the energy storage agent are constructed, and the bidding neural network is established using the strategy function network and the value function network; the bidding time of the energy storage agent is obtained, and the bidding time is trigonometrically encoded to obtain the corresponding encoding result; the market price of the energy storage agent in the target time period is collected, and the market price is discrete Fourier transformed to obtain the corresponding discrete Fourier transform result, and the angle and amplitude values ​​of the first three dimensions in the discrete Fourier transform result are extracted; the state of charge information and market price corresponding to the energy storage agent are obtained, and the environmental information is determined according to the state of charge information, the encoding result, the angle and the amplitude value; the environmental information and the market price are input into the bidding neural network to output the self-dispatching power of the target energy storage agent in the target electricity market.

[0085] In the actual implementation process, the embodiments of the present application can use a common multilayer perceptron network to construct a strategy function network and a value function network for energy storage agent bidding, wherein the multilayer perceptron network is a superposition of M single-layer fully connected neural networks with N nodes, and all single-layer fully connected networks are connected through the Tanh activation function. Since the output range of Tanh is (-1,1), the output range of the neural network is also (-1,1).

[0086] It should be noted that the input of the bidding neural network in the embodiment of the present application consists of two parts: environmental information and market price, as described below:

[0087] 1. Environmental information:

[0088] Environmental information mainly includes the following three aspects of data:

[0089] (1) The time of day when bidding (i.e., bidding time) is encoded using trigonometric functions: sin(T / 24), cos(T / 24) to obtain the corresponding encoding result, which is used to indicate the time of day;

[0090] (2) Market history t , which includes the angles and amplitudes of the first three dimensions of the discrete Fourier transform of the market prices in the past 6 hours and the market prices in the past 4 days (i.e., the market prices in the target time period);

[0091] (3) State of Charge SoC t .

[0092] 2. Market price λ t :

[0093] During the training process of the embodiment of the present application, the energy storage can obtain the market price as information in the bidding process, and the market price is the real price value.

[0094] Therefore, the embodiment of the present application can input environmental information and market prices into the bidding neural network to output the power p of the energy storage self-dispatched in the power market. t , where the original output range of the neural network is (-1,1), which will be mapped to the power range of the energy storage (p min ,p max ).

[0095] According to the bidding method for energy storage quotation and quotation participating in the electricity spot market in the offline training stage proposed in the embodiment of the present application, a state space model and a reward function model of the target energy storage entity operating in the target electricity market are constructed; a price-responsive bidding neural network of the energy storage intelligent body corresponding to the target energy storage entity in the target electricity market price environment is constructed, and the environmental information and market price of the target energy storage entity are obtained, and the environmental information and market price are input into the bidding neural network to output the corresponding self-scheduling power; based on the state space model, the reward function model and the preset PPO algorithm, and in combination with the environmental information, the market price and the self-scheduling power, the bidding neural network is trained to obtain the target energy storage bidding neural network model, thereby realizing effective training of the bidding algorithm.

[0096] Figure 2 A flowchart of a bidding method for energy storage quotation and participation in the electricity spot market during the online deployment phase provided in an embodiment of the present application.

[0097] like Figure 2 As shown, the bidding method for energy storage quotation to participate in the electricity spot market includes the following steps:

[0098] In step S201, multiple market electricity prices corresponding to the preset energy storage entity are obtained, and each of the multiple market electricity prices is input into a pre-constructed target energy storage bidding neural network model in turn to output the energy storage unit power corresponding to each market electricity price, wherein the target energy storage bidding neural network model is obtained by training a preset bidding neural network with a pre-state space model, a reward function model and a PPO algorithm.

[0099] In step S202, the energy storage unit power output by the target energy storage bidding neural network model and the market electricity price corresponding to the energy storage unit power are uniformly sampled to obtain corresponding multiple associated sampling results, and a unit supply curve is constructed according to the multiple associated sampling results.

[0100] In the online deployment stage, the embodiment of the present application can first be based on the price response neural network (i.e., the target energy storage bidding neural network model) constructed and trained as described above, and sample the correlation between electricity price and energy storage unit power. This correlation is a function from price to power, which is recorded as the supply curve of the unit.

[0101] Specifically, the embodiment of the present application is to obtain the lowest market price λ at the input end of the price response neural network. min To the highest market price λ max , with a resolution of M segments, changing the market price λ t The output value of the sampling price response neural network is used to obtain the unit supply curve. The sampled supply curve can be recorded as

[0102] [(λ min ,…,λ max ),(p(λ min ),…,p(λ max ))].

[0103] Therefore, the embodiment of the present application uses a price response neural network to sample from the lowest market price to the highest price at a certain resolution to obtain the corresponding power output, so as to model the relationship between the electricity price and the power of the energy storage unit as a unit supply curve.

[0104] In step S203, the unit supply curve is monotonized and discretized to generate a quantity quotation bid that meets the preset market requirements.

[0105] Furthermore, the embodiments of the present application also need to perform monotonic processing on the sampled unit supply curve to ensure that the power at a higher price is not lower than the power at a lower price, so as to meet the market's requirement for monotonically increasing bid prices; thereafter, the embodiments of the present application need to discretize the monotonic unit supply curve into price-power pairs, and use the initial bid pairs to perform equally spaced sampling on price and power, and adjust through an iterative algorithm until the change ratio of all bid pairs is less than a preset threshold, thereby generating a volume quotation bid that meets market requirements.

[0106] Optionally, in one embodiment of the present application, the unit supply curve is monotonized and discretized to generate a target quantity quotation bid that meets preset market requirements, including: monotonizing the unit supply curve to generate a monotonically increasing unit supply curve that meets the monotonically increasing requirements of the preset market bidding target, and discretizing the monotonically converted unit supply curve to obtain multiple quantity quotation bid pairs; based on a preset initial bidding pair, performing equally spaced sampling in multiple quantity quotation bid pairs to obtain multiple bidding pair sampling data; iteratively calculating the change ratio corresponding to each bidding pair sampling data in the multiple bidding pair sampling data through a preset iterative algorithm until the change ratio corresponding to each bidding pair sampling data is less than a preset threshold value, to generate a target quantity quotation bid.

[0107] Specifically, the embodiment of the present application can firstly make the unit supply curve monotonic, that is, take the monotonic supply curve power as the maximum value of all powers less than or equal to the price, as shown in the following formula, thereby obtaining a monotonic unit supply curve to meet the market's requirement for monotonic increase in bidding targets:

[0108]

[0109] Afterwards, the embodiment of the present application may discretize the monotonic supply curve into price-power pairs (ie, multiple quantity quotation and bid pairs) to generate quantity quotation and bids that meet market requirements.

[0110] In the specific implementation process, the embodiment of the present application can record the final generated quotation and bidding pair as (λ i ,p i ), i=1,…,N; p i The initial value is p max ,λ i The initial value of min ,λ max )(that is, the equally spaced sampling values ​​of the initial bidding pair); the two auxiliary bidding pairs are recorded as (λ 0 ,p 0 )←(λ min ,p min ), (λ N+1 ,p N+1 )←(λ max ,p max ).

[0111] Afterwards, the embodiment of the present application can bid according to the preset iterative quotation quantity (λ i ,p i ) until convergence.

[0112] Optionally, in one embodiment of the present application, the mathematical expression of the preset iterative algorithm is:

[0113]

[0114] Among them, λ i represents the iteratively generated bidding price; p i represents the iteratively generated bidding power; λ k represents the market price in the sampled data of multiple bidding pairs; p k Represents the power in the sampled data for multiple bid pairs.

[0115] It should be noted that the mathematical expression of the iterative algorithm in the embodiment of the present application is as follows:

[0116]

[0117] Among them, λ i represents the iteratively generated bidding price; p i represents the iteratively generated bidding power; λ k represents the market price in the sampled data of multiple bidding pairs (i.e., the market price sampled at equal intervals); p k Represents the power in the sampled data of multiple bidding pairs (i.e., the power corresponding to the equally spaced sampling prices).

[0118] In the embodiment of the present application, the convergence criterion is all (λ i ,p i) in a certain iteration round, the updated change ratio does not exceed the preset threshold ∈, so the generated quantity-price pair (λ i ,p i ) (i.e. target quantity quotation bidding).

[0119] Therefore, the embodiments of the present application can more accurately and effectively release the bidding potential of the quantity quotation format, while giving full play to the advantages of deep reinforcement learning methods in dealing with price uncertainty, so as to flexibly and effectively generate high-dimensional quantity quotation bids.

[0120] According to the bidding method for energy storage quantity quotation participating in the electricity spot market in the online deployment stage proposed in the embodiment of the present application, multiple market electricity prices corresponding to the preset energy storage entity are obtained, and each of the multiple market electricity prices is input into the pre-constructed target energy storage bidding neural network model in turn to output the energy storage unit power corresponding to each market electricity price, wherein the target energy storage bidding neural network model is obtained by training the preset bidding neural network with a pre-state space model, a reward function model and a PPO algorithm; the energy storage unit power output by the target energy storage bidding neural network model and the market electricity price corresponding to the energy storage unit power are uniformly sampled to obtain corresponding multiple associated sampling results, and a unit supply curve is constructed according to the multiple associated sampling results; the unit supply curve is monotonized and discretized to generate a quantity quotation bid that meets the preset market requirements, thereby being able to more accurately and effectively release the bidding potential of the quantity quotation format, while giving full play to the advantages of the deep reinforcement learning method in dealing with price uncertainty.

[0121] Secondly, a bidding device for participating in the electricity spot market by submitting energy storage quotation according to an embodiment of the present application is described with reference to the accompanying drawings.

[0122] Figure 3 It is a block diagram of a bidding device for energy storage quantity quotation and participation in the electricity spot market in an offline training phase according to an embodiment of the present application.

[0123] like Figure 3 As shown, the bidding device 10 for bidding energy storage quantity and price in the electricity spot market includes: a first modeling module 101 , a second modeling module 102 and a training module 300 .

[0124] Among them, the first modeling module 101 is used to construct a state space model and a reward function model of the target energy storage entity operating in the target power market.

[0125] The second modeling module 102 is used to construct a price-responsive bidding neural network for the energy storage intelligent entity corresponding to the target energy storage entity under the target electricity market price environment, and obtain the environmental information and market price of the target energy storage entity, and input the environmental information and market price into the bidding neural network to output the corresponding self-dispatching power.

[0126] The training module 103 is used to train the bidding neural network based on the state space model, the reward function model and the preset PPO algorithm, and in combination with environmental information, market price and self-scheduling power to obtain a target energy storage bidding neural network model.

[0127] Optionally, in one embodiment of the present application, the first modeling module 101 includes: a first acquisition unit, a second acquisition unit, a first determination unit and a construction unit.

[0128] Among them, the first acquisition unit is used to obtain the energy market clearing price in the target power market, and determine the market bidding income according to the energy market clearing price.

[0129] The second acquisition unit is used to acquire the discharge power and energy storage depreciation cost parameters corresponding to the target energy storage entity, and calculate the energy storage depreciation cost through the discharge power and energy storage depreciation cost parameters.

[0130] The first determination unit is used to determine the SoC out-of-bounds penalty corresponding to the target energy storage entity according to the state of charge out-of-bounds penalty constant and the maximum state of charge corresponding to the target energy storage entity.

[0131] A building block is used to construct a reward function model based on market bidding revenue, energy storage depreciation cost, and SoC out-of-bounds penalty.

[0132] Optionally, in one embodiment of the present application, the second modeling module 102 includes: a building unit, a coding unit, a transformation unit, a second determination unit and an input unit.

[0133] Among them, a unit is established to construct a strategy function network and a value function network for energy storage agent bidding based on a preset multi-layer perceptron network, and to establish a bidding neural network using the strategy function network and the value function network.

[0134] The encoding unit is used to obtain the bidding time of the energy storage intelligent body and perform trigonometric function encoding on the bidding time to obtain a corresponding encoding result.

[0135] The transformation unit is used to collect the market price of the energy storage intelligent body in the target time period, and perform discrete Fourier transform on the market price to obtain the corresponding discrete Fourier transform result, and extract the angle and amplitude values ​​of the first three dimensions in the discrete Fourier transform result.

[0136] The second determination unit is used to obtain the state of charge information and market price corresponding to the energy storage intelligent body, and determine the environmental information according to the state of charge information, encoding results, angles and amplitude values.

[0137] The input unit is used to input environmental information and market prices into the bidding neural network to output the self-dispatching power of the target energy storage entity in the target power market.

[0138] It should be noted that the aforementioned explanation of the embodiment of the bidding method for energy storage quantity quotation participating in the electricity spot market during the offline training stage is also applicable to the bidding device for energy storage quantity quotation participating in the electricity spot market during the embodiment of the offline training stage, and will not be repeated here.

[0139] According to the bidding device for energy storage quantity quotation and participation in the electricity spot market in the offline training stage proposed in the embodiment of the present application, the device includes a first modeling module for constructing a state space model and a reward function model of a target energy storage entity operating in a target electricity market; a second modeling module for constructing a price-responsive bidding neural network of an energy storage intelligent entity corresponding to the target energy storage entity under a target electricity market price environment, and obtaining environmental information and market prices of the target energy storage entity, and inputting the environmental information and market prices into the bidding neural network to output the corresponding self-scheduling power; a training module for training the bidding neural network based on the state space model, the reward function model and the preset PPO algorithm, and in combination with the environmental information, the market price and the self-scheduling power to obtain a target energy storage bidding neural network model, thereby achieving effective training of the bidding algorithm.

[0140] Figure 4 It is a block diagram of a bidding device for energy storage quotation and participation in the electricity spot market in an online deployment phase according to an embodiment of the present application.

[0141] like Figure 4 As shown, the bidding device 20 for energy storage quotation and participation in the electricity spot market during the online deployment phase includes: a deployment module 201 , a sampling module 202 and a generation module 203 .

[0142] Among them, the deployment module 201 is used to obtain multiple market electricity prices corresponding to the preset energy storage entities, and input each of the multiple market electricity prices into the pre-constructed target energy storage bidding neural network model in turn to output the energy storage unit power corresponding to each market electricity price, wherein the target energy storage bidding neural network model is obtained by training the preset bidding neural network with a pre-state space model, a reward function model and a PPO algorithm.

[0143] The sampling module 202 is used to uniformly sample the energy storage unit power output by the target energy storage bidding neural network model and the market electricity price corresponding to the energy storage unit power to obtain corresponding multiple associated sampling results, and construct a unit supply curve based on the multiple associated sampling results.

[0144] The generation module 203 is used to perform monotonicity and discretization operations on the unit supply curve to generate a quantity quotation bid that meets the preset market requirements.

[0145] Optionally, in one embodiment of the present application, the generation module 203 includes: a monotonization unit, a processing unit and an iteration unit.

[0146] Among them, the monotonicization unit is used to perform monotonicization operations on the unit supply curve to generate a monotonic unit supply curve that meets the monotonic increasing requirements of the preset market bidding target, and discretize the monotonic unit supply curve to obtain multiple quotation and bidding pairs.

[0147] The processing unit is used to perform equal-interval sampling in multiple bidding pairs based on a preset initial bidding pair to obtain multiple bidding pair sampling data.

[0148] The iteration unit is used to iteratively calculate the change ratio corresponding to each bid pair sampling data in multiple bid pair sampling data through a preset iteration algorithm until the change ratio corresponding to each bid pair sampling data is less than a preset threshold value, so as to generate a target quotation bid.

[0149] Optionally, in one embodiment of the present application, the mathematical expression of the preset iterative algorithm is:

[0150]

[0151] Among them, λ i represents the iteratively generated bidding price; p i represents the iteratively generated bidding power; λ k represents the market price in the sampled data of multiple bidding pairs; p k Represents the power in the sampled data for multiple bid pairs.

[0152] It should be noted that the aforementioned explanation of the embodiment of the bidding method for energy storage quantity quotation to participate in the electricity spot market during the online deployment stage is also applicable to the bidding device for energy storage quantity quotation to participate in the electricity spot market during the online deployment stage embodiment, and will not be repeated here.

[0153] According to the bidding device for energy storage quantity quotation and quotation participating in the electricity spot market in the online deployment stage proposed in the embodiment of the present application, the device includes a deployment module for obtaining multiple market electricity prices corresponding to preset energy storage entities, and inputting each of the multiple market electricity prices into a pre-constructed target energy storage bidding neural network model in turn to output the energy storage unit power corresponding to each market electricity price, wherein the target energy storage bidding neural network model is obtained by training the preset bidding neural network with a pre-state space model, a reward function model and a PPO algorithm; a sampling module for uniformly sampling the energy storage unit power output by the target energy storage bidding neural network model and the market electricity price corresponding to the energy storage unit power to obtain corresponding multiple associated sampling results, and constructing a unit supply curve according to the multiple associated sampling results; a generation module for performing monotonicity and discretization operations on the unit supply curve to generate a quantity quotation and quotation bid that meets the preset market requirements, thereby being able to more accurately and effectively release the bidding potential of the quantity quotation format, while giving full play to the advantages of the deep reinforcement learning method in dealing with price uncertainty.

[0154] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0155] A memory 501 , a processor 502 , and a computer program stored in the memory 501 and executable on the processor 502 .

[0156] When the processor 502 executes the program, the bidding method for energy storage quotation to participate in the electricity spot market provided in the above embodiment is implemented.

[0157] Furthermore, the electronic device further comprises:

[0158] The communication interface 503 is used for communication between the memory 501 and the processor 502 .

[0159] The memory 501 is used to store computer programs that can be executed on the processor 502 .

[0160] The memory 501 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0161] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the communication interface 503, the memory 501 and the processor 502 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0162] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0163] The processor 502 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0164] The embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for bidding for energy storage quantity and quotation in the electricity spot market.

[0165] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0166] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0167] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0168] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0169] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0170] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0171] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0172] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A bidding method for energy storage quotation to participate in the electricity spot market, applied in the offline training stage, characterized in that: The following steps are involved: Construct the state space model and reward function model of the target energy storage entity operating in the target power market; Constructing a price-responsive bidding neural network for the energy storage agent corresponding to the target energy storage subject under the target power market price environment, and obtaining environmental information and market price of the target energy storage subject, and inputting the environmental information and market price into the bidding neural network to output the corresponding self-dispatching power; Based on the state space model, the reward function model and the preset PPO algorithm, and in combination with the environmental information, the market price and the self-scheduling power, the bidding neural network is trained to obtain a target energy storage bidding neural network model.

2. The method according to claim 1, characterized in that The constructing of a state space model and a reward function model of the target energy storage subject operating in the target power market includes: Obtaining an energy market clearing price in the target power market, and determining a market bidding income according to the energy market clearing price; Obtaining the discharge power and energy storage depreciation cost parameters corresponding to the target energy storage entity, and calculating the energy storage depreciation cost through the discharge power and the energy storage depreciation cost parameters; Determine the SoC out-of-bounds penalty corresponding to the target energy storage subject according to the state of charge out-of-bounds penalty constant and the maximum state of charge corresponding to the target energy storage subject; A reward function model is constructed based on the market bidding income, the energy storage depreciation cost and the SoC out-of-bounds penalty.

3. The method according to claim 1, characterized in that The step of constructing a price-responsive bidding neural network for the energy storage intelligent entity corresponding to the target energy storage entity under the target power market price environment, obtaining environmental information and market price of the target energy storage entity, and inputting the environmental information and market price into the bidding neural network to output the corresponding self-dispatching power includes: Based on a preset multi-layer perceptron network, construct a strategy function network and a value function network for bidding of the energy storage agent, and use the strategy function network and the value function network to establish the bidding neural network; Obtaining the bidding time of the energy storage intelligent entity, and performing trigonometric function encoding on the bidding time to obtain a corresponding encoding result; Collect the market price of the energy storage intelligent body in the target time period, and perform discrete Fourier transform on the market price to obtain a corresponding discrete Fourier transform result, and extract the angle and amplitude values ​​of the first three dimensions in the discrete Fourier transform result; Acquire the state of charge information and the market price corresponding to the energy storage intelligent body, and determine the environmental information according to the state of charge information, the encoding result, the angle and the amplitude value; The environmental information and the market price are input into the bidding neural network to output the self-dispatching power of the target energy storage entity in the target electricity market.

4. A bidding method for energy storage quotation to participate in the electricity spot market, applied in the online deployment stage, characterized in that: The following steps are involved: Obtain multiple market electricity prices corresponding to preset energy storage entities, and input each of the multiple market electricity prices into a pre-constructed target energy storage bidding neural network model in turn to output the energy storage unit power corresponding to each market electricity price, wherein the target energy storage bidding neural network model is obtained by training a preset bidding neural network using a pre-state space model, a reward function model, and a PPO algorithm; Uniformly sampling the energy storage unit power output by the target energy storage bidding neural network model and the market electricity price corresponding to the energy storage unit power to obtain a corresponding plurality of associated sampling results, and constructing a unit supply curve according to the plurality of associated sampling results; The unit supply curve is monotonized and discretized to generate a quantity quotation bid that meets preset market requirements.

5. The method according to claim 4, characterized in that The monotonicization and discretization operations are performed on the unit supply curve to generate a target quotation bid that meets the preset market requirements, including: Performing a monotonic operation on the unit supply curve to generate a monotonic unit supply curve that meets the monotonic increasing requirement of a preset market bidding target, and performing a discretization operation on the monotonic unit supply curve to obtain a plurality of quotation and bid pairs; Based on the preset initial bidding pairs, sampling is performed at equal intervals in the multiple bidding pairs to obtain multiple bidding pair sampling data; The change ratio corresponding to each bid pair sampling data in the multiple bid pair sampling data is iteratively calculated through a preset iterative algorithm until the change ratio corresponding to each bid pair sampling data is less than a preset threshold value, so as to generate the target quantity quotation bid.

6. The method according to claim 5, characterized in that The mathematical expression of the preset iterative algorithm is: Among them, λ i represents the iteratively generated bidding price; p i represents the iteratively generated bidding power; λ k represents the market price in the sample data of the plurality of bid pairs; k The power in the sampled data of the plurality of bidding pairs is represented.

7. A bidding device for energy storage quotation to participate in the electricity spot market, applied in the offline training stage, characterized in that: include: The first modeling module is used to construct a state space model and a reward function model of the target energy storage entity operating in the target power market; The second modeling module is used to construct a price-responsive bidding neural network of the energy storage intelligent body corresponding to the target energy storage subject under the target power market price environment, and obtain the environmental information and market price of the target energy storage subject, and input the environmental information and the market price into the bidding neural network to output the corresponding self-dispatching power; A training module is used to train the bidding neural network based on the state space model, the reward function model and the preset PPO algorithm, and in combination with the environmental information, the market price and the self-scheduling power to obtain a target energy storage bidding neural network model.

8. A bidding device for energy storage quotation to participate in the electricity spot market, applied in the online deployment stage, characterized in that: include: A deployment module is used to obtain multiple market electricity prices corresponding to preset energy storage entities, and input each of the multiple market electricity prices into a pre-built target energy storage bidding neural network model in turn to output the energy storage unit power corresponding to each market electricity price, wherein the target energy storage bidding neural network model is obtained by training a preset bidding neural network with a pre-state space model, a reward function model and a PPO algorithm; A sampling module, used for uniformly sampling the energy storage unit power output by the target energy storage bidding neural network model and the market electricity price corresponding to the energy storage unit power, so as to obtain a corresponding plurality of associated sampling results, and construct a unit supply curve according to the plurality of associated sampling results; A generation module is used to perform monotonicity and discretization operations on the unit supply curve to generate a quantity quotation bid that meets preset market requirements.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the bidding method for participating in the electricity spot market by quoting energy storage quantity as described in any one of claims 1-3 or 4-5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement a bidding method for energy storage quotation to participate in the electricity spot market as described in any one of claims 1-3 or 4-5.

Images