Power market bidding method based on distributed robust strategy

Abstract

The invention relates to an electricity market bidding method based on a distributed robust strategy, and belongs to the technical field of electricity markets. Comprising the following steps: representing a state space of a time slot t by using a market clearing electricity price of the time slot t, and representing an action space of the time slot t by using a bidding electricity price of a kth bidding participant in the time slot t; constructing a reward function based on the state space of the time slot t, the state space of the time slot t + 1, the action space of the time slot t, the predicted bidding strategies of other bidding participants except the kth bidding participant in the time slot t and the income of the kth bidding participant in the time slot t under the potential risk; constructing an action value function that the k-th bidding participant obeys the bidding strategy; and solving by using a DRMA DDPG algorithm to obtain an optimal bidding strategy of the kth bidding participant. According to the method, the uncertainty factors in the electricity market bidding process are fully considered, so that the finally obtained optimal bidding strategy can maximize the income of the bidding participants.

Description

Technical Field

[0001] The invention relates to the technical field of power market, and in particular to a power market bidding method of a distributed robust strategy. Background Art

[0002] Under the background of power system reform, the construction of power market is imperative. The primary problem faced by power market participants is how to give the best bidding strategy to maximize profits.

[0003] The existing power market bidding methods generally assume that market players can obtain global market information, that is, understand the bidding strategies of all bidding participants in the market. However, in the actual bidding process, bidders can only master their own bidding strategies and information, and the bidding strategies and information of other bidding participants are often confidential and unavailable. Therefore, the existing power market bidding methods do not consider the impact of the uncertainty of the bidding strategies of other bidding participants on the optimal bidding strategy, and cannot optimize their own bidding strategies based on the bidding strategies of other bidding participants, resulting in low accuracy of the obtained optimal bidding strategy.

[0004] In summary, the existing electricity market bidding methods do not take into account the uncertainty of the bidding strategies of other bidding participants, and are unable to optimize their own bidding strategies based on the bidding strategies of other bidding participants, resulting in low accuracy of the optimal bidding strategy. Summary of the invention

[0005] To this end, the technical problem to be solved by the present invention is to overcome the problem that the electricity market bidding method in the prior art does not take into account the uncertainty of the bidding strategies of other bidding participants, and cannot optimize its own bidding strategy based on the bidding strategies of other bidding participants, resulting in low accuracy of the obtained optimal bidding strategy.

[0006] In order to solve the above technical problems, the present invention provides a distributed robust strategy power market bidding method, comprising:

[0007] The process of obtaining the optimal bidding strategy in the electricity market is described as a Markov decision process. The state space, action space and reward function are established, and a deep reinforcement learning framework is obtained, including:

[0008] The state space s of time slot t is represented by the market clearing electricity price of time slot t t , the action space of time slot t is represented by the bid price of the kth bidding participant in time slot t

[0009] The state space s based on time slot t t , the state space s of time slot t+1 t+1 , the action space of time slot t The predicted bidding strategy function of other bidding participants except the k-th bidding participant in time slot t and the reward function constructed by the profit of the k-th bidding participant under potential risk in time slot t

[0010] Construct the kth bidding participant to obey the bidding strategy function In status t and actions The action value function under

[0011] The DRMA DDPG algorithm is used to solve the optimal bidding strategy of the kth bidding participant.

[0012] In one embodiment of the present invention, the reward function for:

[0013]

[0014] in, represents the profit of the k-th bidding participant under potential risk in time slot t, K represents the number of bidding participants, -k represents other bidding participants except the k-th bidding participant, represents the predicted bidding strategy function of all bidding participants except the k-th bidding participant in time slot t, represents the predicted bid electricity price of other bidding participants except the kth bidding participant in time slot t, and γ represents the attenuation coefficient.

[0015] In one embodiment of the present invention, the action value function for:

[0016]

[0017] Among them, μ 0,k represents the probability distribution of the uncertainty set of the net load forecast error, represents the profit of the kth bidding participant under potential risk in time slot t, and K represents the number of bidding participants.

[0018] In one embodiment of the present invention, the method of obtaining the optimal bidding strategy of the kth bidding participant by using the DRMA DDPG algorithm includes:

[0019] Step 1: Randomly initialize the parameters of the actor network and the parameters of the critic network And create a playback cache

[0020] Step 2: Initialize the training segment d to 1;

[0021] Step 3: Initialize the time slot t in segment d to 1;

[0022] Step 4: Set the state s t Input the cator network, output So that the kth bidding participant performs an action and get rewards At the same time, transition to the next state s t+1 , get the tuple The tuple Store to the playback buffer middle;

[0023] Step 5: Determine the playback buffer Is the number of tuples in less than N? If less than N, then t=t+1, and return to step 4 until the playback cache is The number of tuples in is greater than or equal to N;

[0024] Step 6: Initialize the time slot t in segment d to 1;

[0025] Step 7: Replay the video from the playback buffer A sample package consisting of N tuples is randomly selected from the network, and each tuple in the sample package is input into the actor network and the critic network;

[0026] Step 8: For the i-th tuple in the sample package Will s i ′ Input the actor network, output action Will s i ′ and Input the critic network and output the action value function Calculate target value Will s i and Input to the critic network output action value function And calculate the loss of the i-th tuple

[0027] Step 9: Input all tuples in the sample packet into the actor network and the critic network to calculate the penalty function

[0028] Step 10: Update the parameters of the critic network by minimizing the penalty function Update the parameters of the actor network via policy gradients

[0029] Step 11: Determine t <N t Is it true? If so, set t=t+1 and return to step 7;

[0030] Step 12: Determine d <D max Is it true? If true, set d = d + 1 and return to step 3. If not true, get the optimal bidding strategy of the kth bidding participant. Among them, D max is the maximum number of fragments.

[0031] In one embodiment of the present invention, the parameters of the critic network are updated by minimizing the penalty function Update the parameters of the actor network via policy gradients include:

[0032] Adopt Adam optimization method, through gradient Update the parameters of the critic network in, Represents the penalty function L for the parameter The partial derivative of

[0033] Adopt Adam optimization method, through gradient Update the parameters of the actor network

[0034] in, The action value function approximated by the critic network is calculated as:

[0035]

[0036] in, Represents the action-value function right The partial guide, Represents the bidding strategy function μ k Parameters The partial derivative of .

[0037] In one embodiment of the present invention, the parameters of the critic network are The update formula is:

[0038]

[0039] in, represents the parameters of the updated critic network, represents the parameters of the critic network before updating, and ρ represents the network update hyperparameter.

[0040] In one embodiment of the present invention, the parameters of the actor network are The update formula is:

[0041]

[0042] in, represents the parameters of the updated actor network, represents the parameters of the actor network before updating, and ρ represents the network update hyperparameter.

[0043] In one embodiment of the present invention, the predicted bidding strategy function of the bidding participants other than the k-th bidding participant in the time slot t is:

[0044]

[0045] in, represents the predicted bidding strategy function of all bidding participants except the k-th bidding participant in time slot t, represents the predicted bid price of all bidding participants except the kth bidding participant in time slot t, Express expectations, express The random probability distribution of μ k , λ represents the bid price, H(μ k ) represents the cross entropy of the estimated policy function.

[0046] The present invention also provides a distributed robust strategy power market bidding model, including:

[0047] The Markov decision module is used to describe the process of obtaining the optimal bidding strategy in the power market as a Markov decision process, establish the state space, action space and reward function, and obtain a deep reinforcement learning framework, including:

[0048] The state space s of time slot t is represented by the market clearing electricity price of time slot t t , the action space of time slot t is represented by the bid price of the kth bidding participant in time slot t

[0049] The state space s based on time slot t t , the state space s of time slot t+1 t+1 , the action space of time slot t The predicted bidding strategy function of other bidding participants except the k-th bidding participant in time slot t and the reward function constructed by the profit of the k-th bidding participant under potential risk in time slot t

[0050] Construct the kth bidding participant to obey the bidding strategy function In statust and actions The action value function under

[0051] The strategy optimization module is used to use the DRMA DDPG algorithm to solve the optimal bidding strategy of the kth bidding participant.

[0052] The present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned distributed robust strategy power market bidding method are implemented.

[0053] The distributed robust strategy power market bidding method provided by the present invention describes the process of obtaining the optimal bidding strategy in the power market as a Markov decision process, uses the market clearing electricity price at time slot t to characterize the state space of time slot t, and uses the bid electricity price of the k-th bidding participant at time slot t to characterize the action space of time slot t; constructs a reward function based on the state space of time slot t, the state space of time slot t+1, the action space of time slot t, the predicted bidding strategy functions of other bidding participants except the k-th bidding participant at time slot t, and the benefits of the k-th bidding participant under potential risks at time slot t; constructs an action value function for the k-th bidding participant to obey the bidding strategy; uses the DRMADDPG algorithm to solve and obtain the optimal bidding strategy of the k-th bidding participant; the present application uses the predicted bidding strategy functions of other bidding participants to construct a reward function, and uses DRMA The DDPG algorithm is used to solve the problem, which not only takes into account the impact of other bidding participants' bidding strategies on the target participant's optimal bidding strategy, but also does not need to obtain the bidding strategies and information of other bidding participants. It fully considers the uncertainty factors in the bidding process of the power market, so that the optimal bidding strategy finally obtained can maximize the benefits of the bidding participants. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein

[0055] Figure 1 A flow chart of the power market bidding method of the distributed robust strategy provided in this application;

[0056] Figure 2 A flowchart of the optimal bidding strategy obtained by using the DRMA DDPG algorithm provided in this application;

[0057] Figure 3 Schematic diagram of the critic network parameter update process provided for this application;

[0058] Figure 4A schematic diagram of the actor network parameter update process provided for this application;

[0059] Figure 5 A schematic diagram comparing the optimal bidding strategy obtained by the existing method provided in the embodiment of the present application and the optimal bidding strategy obtained by the method of the present application; wherein (a) in the figure is a schematic diagram of the optimal bidding strategy result obtained using the existing method, and (b) in the figure is a schematic diagram of the optimal bidding strategy result obtained using the method provided in the present application. DETAILED DESCRIPTION

[0060] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0061] This application provides a distributed robust strategy for bidding in the electricity market. Figure 1 As shown, it specifically includes:

[0062] S10: Describe the process of obtaining the optimal bidding strategy in the electricity market as a Markov decision process, establish the state space, action space and reward function, and obtain a deep reinforcement learning framework, including:

[0063] S100: Using the market clearing electricity price of time slot t to represent the state space s of time slot t t , the action space of time slot t is represented by the bid price of the kth bidding participant in time slot t

[0064] S101: State space s based on time slot t t , the state space s of time slot t+1 t+1 , the action space of time slot t The predicted bidding strategy function of other bidding participants except the k-th bidding participant in time slot t and the reward function constructed by the profit of the k-th bidding participant under potential risk in time slot t

[0065]

[0066] S102: Construct the bidding strategy function that the kth bidding participant obeys In status t and actions The action value function under

[0067] S20: Use the DRMA DDPG algorithm to solve the optimal bidding strategy of the kth bidding participant.

[0068] The distributed robust strategy for electricity market bidding method provided in the present application describes the process of obtaining the optimal bidding strategy in the electricity market as a Markov decision process, using the market clearing electricity price at time slot t to characterize the state space of time slot t, and using the bid electricity price of the k-th bidding participant at time slot t to characterize the action space of time slot t; constructing a reward function based on the state space of time slot t, the state space of time slot t+1, the action space of time slot t, the predicted bidding strategy functions of other bidding participants except the k-th bidding participant at time slot t, and the return of the k-th bidding participant at time slot t under potential risks; constructing an action value function for the k-th bidding participant to obey the bidding strategy; using the DRMA DDPG algorithm to obtain the optimal bidding strategy of the k-th bidding participant; using the predicted bidding strategy functions of other bidding participants to construct a reward function, and using DRMA The DDPG algorithm is used to solve the problem, which not only takes into account the impact of other bidding participants' bidding strategies on the target participant's optimal bidding strategy, but also does not need to obtain the bidding strategies and information of other bidding participants. It fully considers the uncertainty factors in the bidding process of the power market, so that the optimal bidding strategy finally obtained can maximize the benefits of the bidding participants.

[0069] Specifically, the DRMA DDPG (Distributed Robust Agent Deep Deterministic Policy Gradient) algorithm in this application extends the traditional DDPG algorithm to a multi-agent environment (i.e., multiple bidding participants), and then incorporates the concept of robust Nash equilibrium (RNE) into the formulation of policy gradient and action-value functions, making the algorithm risk-averse and fully considering the impact between the bidding strategies of multiple bidding participants and the uncertainty factors in the bidding process.

[0070] Robust Nash equilibrium is a concept of solution in game theory. In non-cooperative games, bidding entities eventually reach an equilibrium state in order to maximize their own interests. That is, if other bidding entities do not change their strategies, no bidding participant will change his or her bidding strategy. It is used to study the relationship between individual behavior and overall market performance in an uncertain environment.

[0071] Specifically, since the bidding strategies of other bidding participants are kept confidential during the bidding process, the application uses an estimated policy function (EPF) to predict the predicted bidding strategy functions of other bidding participants, and approximates the bidding strategies of other participants by maximizing the logarithmic probability of the behavior of the k-th bidding participant. Based on this, the predicted bidding strategy functions of other bidding participants except the k-th bidding participant in time slot t in step S101 are:

[0072]

[0073] in, represents the predicted bidding strategy function of all bidding participants except the k-th bidding participant in time slot t, represents the predicted bid price of all bidding participants except the kth bidding participant in time slot t, Express expectations, express The random probability distribution of μ k , λ represents the bid price, H(μ k ) represents the cross entropy of the estimated policy function.

[0074] Specifically, in some embodiments of the present application, the reward function in step S101 is for:

[0075]

[0076] in, represents the profit of the k-th bidding participant under potential risk in time slot t, K represents the number of bidding participants, -k represents other bidding participants except the k-th bidding participant, represents the predicted bidding strategies of other bidding participants except the k-th bidding participant in time slot t, represents the predicted bid electricity price of other bidding participants except the kth bidding participant in time slot t, and γ represents the attenuation coefficient.

[0077] Specifically, the profit of the kth bidding participant under potential risk in time slot t is the observed data fed back to the bidding participant by the power market.

[0078] Furthermore, the action value function in step S102 for:

[0079]

[0080] Among them, μ 0,k represents the probability distribution of the uncertainty set of the net load forecast error, represents the profit of the kth bidding participant under potential risk in time slot t, and K represents the number of bidding participants.

[0081] Specifically, since the present application considers bidding decisions in an uncertain environment, the action-value function in the present application is improved from the Bellman equation in the traditional deterministic environment. In addition, the present application also sets the probability distribution of the uncertain set of net load prediction errors, so that the optimal bidding decision finally obtained is robust in any environment.

[0082] In some embodiments of the present application, Figure 2In step S20 shown in FIG. , the DRMA DDPG algorithm is used to solve the optimal bidding strategy of the k-th bidding participant, including:

[0083] Step 1: Randomly initialize the parameters of the actor network and the parameters of the critic network And create a playback cache

[0084] Step 2: Initialize the training segment d to 1;

[0085] Step 3: Initialize the time slot t in segment d to 1;

[0086] Step 4: Set the state s t Input cator network, output So that the kth bidding participant performs an action and get rewards At the same time, transition to the next state s t+1 , get the tuple The tuple Save to playback cache middle;

[0087] Step 5: Determine the playback cache Is the number of tuples in less than N? If so, then t = t + 1, and return to step 4 until the replay cache is reached. The number of tuples in is greater than or equal to N;

[0088] Step 6: Initialize the time slot t in segment d to 1;

[0089] Step 7: Replay Cache A sample package consisting of N tuples is randomly selected from the network, and each tuple in the sample package is input into the actor network and the critic network;

[0090] Step 8: For the i-th tuple in the sample bag Will s i 'Input actor network, output action Will s i 'and Input critic network, output action value function Calculate target value Will s i and Input to the critic network output action value function And calculate the loss of the i-th tuple

[0091] Step 9: Input all tuples in the sample package into the actor network and critic network to calculate the penalty function

[0092] Step 10: Update the parameters of the critic network by minimizing the penalty function Update the parameters of the actor network through policy gradients

[0093] Step 11: Determine t <N t Is it true? If so, set t=t+1 and return to step 7;

[0094] Step 12: Determine d <D max Is it true? If true, set d = d + 1 and return to step 3. If not true, get the optimal bidding strategy of the kth bidding participant. Among them, D max is the maximum number of fragments.

[0095] Specifically, in some embodiments of the present application, the specific implementation of step 10 is:

[0096] Adopt Adam optimization method, through gradient Update the parameters of the critic network in, Represents the penalty function L for the parameter The partial derivative of

[0097] Adopt Adam optimization method, through gradient Update the parameters of the actor network

[0098] in, The action value function approximated by the critic network is calculated as:

[0099]

[0100] in, Represents the action-value function right The partial guide, Represents the bidding strategy function μ k Parameters The partial derivative of .

[0101] Specifically, the parameters of the critic network The update formula is:

[0102]

[0103] in, represents the parameters of the updated critic network, represents the parameters of the critic network before updating, and ρ represents the network update hyperparameter.

[0104] Parameters of the actor network The update formula is:

[0105]

[0106] in, represents the parameters of the updated actor network, represents the parameters of the actor network before updating, and ρ represents the network update hyperparameter.

[0107] Based on the distributed robust strategy power market bidding method provided in the above embodiment, the embodiment of the present application also provides a distributed robust strategy power market bidding model, which specifically includes:

[0108] The Markov decision module 10 is used to describe the process of obtaining the optimal bidding strategy in the power market as a Markov decision process, establish the state space, action space and reward function, and obtain a deep reinforcement learning framework, including:

[0109] The state space s of time slot t is represented by the market clearing electricity price of time slot t t , the action space of time slot t is represented by the bid price of the kth bidding participant in time slot t

[0110] The state space s based on time slot t t , the state space s of time slot t+1 t+1 , the action space of time slot t The predicted bidding strategy function of other bidding participants except the k-th bidding participant in time slot t and the reward function constructed by the profit of the k-th bidding participant under potential risk in time slot t

[0111] Construct the kth bidding participant to obey the bidding strategy function In status t and actions

[0112] The action value function under

[0113] The strategy optimization module 20 is used to solve by using the DRMA DDPG algorithm to obtain the optimal bidding strategy of the kth bidding participant.

[0114] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned distributed robust strategy power market bidding method are implemented.

[0115] In order to verify the effectiveness of the power market bidding method provided by the present application, the embodiment of the present application also provides a bidding strategy convergence process of each bidding participant in the IEEE 3-bus system using the method of the present application and the Q-Learning method in the prior art, such as Figure 5 shown.

[0116] Figure 5 (a) shows the bidding strategy convergence process of each bidding participant in the IEEE 3-bus system using the Q-Learning method; Figure 5 (b) shows the bidding strategy convergence process of each bidding participant in the IEEE 3-bus system of the method provided by the present application; as can be seen from the figure, the method provided by the present application can reach convergence within 6000 episodes, while the Q-Learning method cannot converge.

[0117] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0118] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0119] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0121] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A distributed robust strategy bidding method for electricity market, characterized in that: include: The process of obtaining the optimal bidding strategy in the electricity market is described as a Markov decision process. The state space, action space and reward function are established, and a deep reinforcement learning framework is obtained, including: The state space s of time slot t is represented by the market clearing electricity price of time slot t t , the action space of time slot t is represented by the bid price of the kth bidding participant in time slot t The state space s based on time slot t t , the state space s of time slot t+1 t+1 , the action space of time slot t The predicted bidding strategy function of other bidding participants except the k-th bidding participant in time slot t and the reward function constructed by the profit of the k-th bidding participant under potential risk in time slot t Construct the kth bidding participant to obey the bidding strategy function In status t and actions The action value function under The DRMA DDPG algorithm is used to solve the optimal bidding strategy of the kth bidding participant.

2. The power market bidding method of the distributed robust strategy according to claim 1 is characterized in that: The reward function for: in, represents the profit of the k-th bidding participant under potential risk in time slot t, K represents the number of bidding participants, -k represents other bidding participants except the k-th bidding participant, represents the predicted bidding strategy function of all bidding participants except the k-th bidding participant in time slot t, represents the predicted bid electricity price of other bidding participants except the kth bidding participant in time slot t, and γ represents the attenuation coefficient.

3. The power market bidding method of the distributed robust strategy according to claim 2 is characterized in that: The action-value function for: Among them, μ 0,k represents the probability distribution of the uncertainty set of the net load forecast error, represents the profit of the kth bidding participant under potential risk in time slot t, and K represents the number of bidding participants.

4. The power market bidding method of the distributed robust strategy according to claim 3 is characterized in that: The method of using the DRMA DDPG algorithm to obtain the optimal bidding strategy of the kth bidding participant includes: Step 1: Randomly initialize the parameters of the actor network and the parameters of the critic network And create a playback cache Step 2: Initialize the training segment d to 1; Step 3: Initialize the time slot t in segment d to 1; Step 4: Set the state s t Input the cator network, output So that the kth bidding participant performs an action and get rewards At the same time, transition to the next state s t+1 , get the tuple The tuple Store to the playback buffer middle; Step 5: Determine the playback buffer Is the number of tuples in less than N? If less than N, then t=t+1, and return to step 4 until the playback cache is The number of tuples in is greater than or equal to N; Step 6: Initialize the time slot t in segment d to 1; Step 7: Replay the video from the playback buffer A sample package consisting of N tuples is randomly selected from the network, and each tuple in the sample package is input into the actor network and the critic network; Step 8: For the i-th tuple in the sample package Will s i ′ Input the actor network, output action Will s i ′ and Input the critic network and output the action value function Calculate target value Will s i and Input to the critic network output action value function And calculate the loss of the i-th tuple Step 9: Input all tuples in the sample packet into the actor network and the critic network to calculate the penalty function Step 10: Update the parameters of the critic network by minimizing the penalty function Update the parameters of the actor network via policy gradients Step 11: Determine t <N t Is it true? If so, set t=t+1 and return to step 7; Step 12: Determine d <D max Is it true? If true, set d = d + 1 and return to step 3. If not true, get the optimal bidding strategy of the kth bidding participant. Among them, D max is the maximum number of fragments.

5. The power market bidding method of the distributed robust strategy according to claim 4 is characterized in that: The parameters of the critic network are updated by minimizing the penalty function Update the parameters of the actor network via policy gradients include: Adopt Adam optimization method, through gradient Update the parameters of the critic network in, Represents the penalty function L for the parameter The partial derivative of Adopt Adam optimization method, through gradient Update the parameters of the actor network in, The action value function approximated by the critic network is calculated as: in, Represents the action-value function right The partial guide, Represents the bidding strategy function μ k Parameters The partial derivative of .

6. The power market bidding method of the distributed robust strategy according to claim 5 is characterized in that: The parameters of the critic network The update formula is: in, represents the parameters of the updated critic network, represents the parameters of the critic network before updating, and ρ represents the network update hyperparameter.

7. The distributed robust strategy power market bidding method according to claim 5 is characterized in that: Parameters of the actor network The update formula is: in, represents the parameters of the updated actor network, represents the parameters of the actor network before updating, and ρ represents the network update hyperparameter.

8. The distributed robust strategy power market bidding method according to claim 1 is characterized in that: The predicted bidding strategy function of the bidding participants other than the k-th bidding participant in the time slot t is: in, represents the predicted bidding strategy function of all bidding participants except the k-th bidding participant in time slot t, represents the predicted bid price of all bidding participants except the kth bidding participant in time slot t, Express expectations, express The random probability distribution of μ k , λ represents the bid price, H(μ k ) represents the cross entropy of the estimated policy function.

9. A distributed robust strategy power market bidding model, characterized in that: include: The Markov decision module is used to describe the process of obtaining the optimal bidding strategy in the power market as a Markov decision process, establish the state space, action space and reward function, and obtain a deep reinforcement learning framework, including: The state space s of time slot t is represented by the market clearing electricity price of time slot t t , the action space of time slot t is represented by the bid price of the kth bidding participant in time slot t The state space s based on time slot t t , the state space s of time slot t+1 t+1 , the action space of time slot t The predicted bidding strategy function of other bidding participants except the k-th bidding participant in time slot t and the reward function constructed by the profit of the k-th bidding participant under potential risk in time slot t Construct the kth bidding participant to obey the bidding strategy function In status t and actions The action value function under The strategy optimization module is used to use the DRMA DDPG algorithm to solve the optimal bidding strategy of the kth bidding participant.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the distributed robust strategy power market bidding method according to any one of claims 1 to 8 are implemented.

Images