Optimization method and device for new energy system
By creating a scheduling model in the new energy system and optimizing the operation strategies of photovoltaic and pumped storage equipment using deep reinforcement learning algorithms, the grid stability problems caused by intermittent photovoltaic panels and frequent conversion of energy storage equipment are solved, and the stability of the system is achieved Electrical energy output and efficient complement each other.
Patent Information
- Application Number
- CN202411299257.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2024-12-27
AI Technical Summary
The intermittent and instability of photovoltaic panels and the frequent transition of energy storage equipment in the working state have led to a reduction in the stability of the power grid of the new energy system. A scheduling strategy for optimizing the complementary system of photovoltaic and pumped storage is needed to output stable electricity.
Create a scheduling model of the new energy system, model it as a Markov decision-making process (MDP), design state space, action space and reward functions, and use deep reinforcement learning algorithm (DDPG) to obtain optimization strategies by adjusting photovoltaic panels and pumped storage The operation strategy of the energy device is to reduce power fluctuations and improve system stability.
It has achieved 100% absorption and optimization of energy storage equipment operation efficiency of photovoltaic power generation, reduced power fluctuations in the connection point, and improved the economic and stability of the system.
Smart Images
Figure CN119209744A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of energy storage technology, and in particular to an optimization method and device for a new energy system. Background Art
[0002] With the development of technology and the progress of society, new energy sources have been widely used. Among them, there is a new energy system that is equipped with photovoltaic panels, pumped storage equipment and loads. In addition, water storage equipment (for example, ponds, etc.) is also provided to match the pumped storage equipment. When water storage and energy storage are needed, water can be pumped into the water storage equipment through the pumped storage equipment, and then the electrical energy will be converted into the potential energy of the water. When the potential energy of the water needs to be converted into electrical energy, the water storage equipment can be opened and a hydroelectric generator can be used to convert the potential energy of the water into electrical energy.
[0003] Understandably, the intermittent and unstable nature of photovoltaic panels poses a challenge to the stable operation of the power grid. Currently, the prediction accuracy of photovoltaic power generation is not high, and the working state of energy storage equipment changes frequently, resulting in reduced system stability. Therefore, a scheduling strategy that can optimize the photovoltaic and pumped storage complementary system in real time is needed to enable the new energy system to output stable electricity. Summary of the invention
[0004] The object of the present invention is to provide an optimization method and device for a new energy system.
[0005] In order to achieve one of the above-mentioned purposes of the invention, an embodiment of the present invention provides an optimization method for a new energy system, wherein the new energy system is provided with a photovoltaic panel, a pumped storage device and a load; the method comprises the following steps: creating a scheduling model corresponding to the new energy system, wherein the scheduling model is used to characterize how to perform power scheduling between the photovoltaic panel and the pumped storage device so that the new energy system outputs stable electric energy; modeling the scheduling model into a Markov decision process MDP, designing its state space, action space and reward function; and using the DDPG algorithm to obtain an optimization strategy for the Markov decision process MDP.
[0006] As a further improvement of an embodiment of the present invention, in the Markov decision process MDP, the state s at time t t At least including: the output power P of the photovoltaic panel PV,t , the water level SOC of the pumped storage equipment t and the load power L t ; In state s t The action taken next t ; Reward function r(s t ,a t )=-α·(ΔP t )2 +β·ΔE t , where ΔP t is the power change, ΔE t is the change in economic benefits, and α and β are weight coefficients.
[0007] As a further improvement of an embodiment of the present invention, action a t At least including: the pumping speed of the pumped storage equipment or the power generation power P of the photovoltaic panel PHS,t .
[0008] As a further improvement of an embodiment of the present invention, the components of the DDPG algorithm include: an Actor network μ(s|θ) for mapping a given state s to an action a μ ), a critic network Q(s,a|θ) for evaluating the value of taking action a in state s Q ), target Actor network μ′(s|θ μ′ ) and the target Critic network Q′(s,a|θ Q′ ); the critic network Q(s,a|θ Q ) is updated based on minimizing the mean square error between the predicted Q value and the target Q value; the Actor network μ(s|θ μ ) is updated based on the policy gradient method.
[0009] As a further improvement of an embodiment of the present invention, the learning rate adjustment formula of the DDPG algorithm is: t =η 0 ×d (floor(t / T)) , where η 0 is the initial learning rate, d is the decay factor, t is the current number of iterations, T is the length of each cycle, and the floor function represents rounding down.
[0010] An embodiment of the present invention also provides an optimization device for a new energy system, wherein the new energy system is provided with photovoltaic panels, pumped storage equipment and loads; the device comprises the following modules: a model creation module, used to create a scheduling model corresponding to the new energy system, the scheduling model is used to characterize how to perform power scheduling between photovoltaic panels and pumped storage equipment, so that the new energy system outputs stable electric energy; an optimization module, used to model the scheduling model into a Markov decision process MDP, design its state space, action space and reward function; and use the DDPG algorithm to obtain an optimization strategy for the Markov decision process MDP.
[0011] As a further improvement of an embodiment of the present invention, in the Markov decision process MDP, the state s at time t tAt least including: the output power P of the photovoltaic panel PV,t , the water level SOC of the pumped storage equipment t and the load power L t ; In state s t The action taken next t ; Reward function r(s t ,a t )=-α·(ΔP t ) 2 +β·ΔE t , where ΔP t is the power change, ΔE t is the change in economic benefits, and α and β are weight coefficients.
[0012] As a further improvement of an embodiment of the present invention, action a t At least including: the pumping speed of the pumped storage equipment or the power generation power P of the photovoltaic panel PHS,t .
[0013] As a further improvement of an embodiment of the present invention, the components of the DDPG algorithm include: an Actor network μ(s|θ) for mapping a given state s to an action a μ ), a critic network Q(s,a|θ) for evaluating the value of taking action a in state s Q ), target Actor network μ′(s|θ μ′ ) and the target Critic network Q′(s,a|θ Q′ ); the critic network Q(s,a|θ Q ) is updated based on minimizing the mean square error between the predicted Q value and the target Q value; the Actor network μ(s|θ μ ) is updated based on the policy gradient method.
[0014] As a further improvement of an embodiment of the present invention, the learning rate adjustment formula of the DDPG algorithm is: t =η 0 ×d (floor(t / T)) , where η 0 is the initial learning rate, d is the decay factor, t is the current number of iterations, T is the length of each cycle, and the floor function represents rounding down.
[0015] Compared with the prior art, the technical effect of the present invention is that: the embodiment of the present invention provides an optimization method and device for a new energy system, and the optimization method includes the following steps: creating a scheduling model corresponding to the new energy system, the scheduling model is used to characterize how to perform power scheduling between photovoltaic panels and pumped storage equipment, so that the new energy system outputs stable electric energy; modeling the scheduling model into a Markov decision process MDP, designing its state space, action space and reward function; using the DDPG algorithm to obtain the optimization strategy of the Markov decision process MDP. The optimization method can enable the new energy system to output stable electric energy. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 It is a flowchart of the optimization method in an embodiment of the present invention. DETAILED DESCRIPTION
[0017] The present invention will be described in detail below in conjunction with the various embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional changes made by a person skilled in the art based on these embodiments are all within the scope of protection of the present invention.
[0018] As used herein, terms indicating spatial relative positions such as "upper," "above," "lower," and "below" are used for the purpose of convenience to describe the relationship of one unit or feature relative to another unit or feature as shown in the accompanying drawings. Terms of spatial relative position may be intended to include different orientations of the device in use or operation in addition to the orientation shown in the drawings. For example, if the device in the figure is turned over, units described as being "below" or "beneath" other units or features will be located "above" the other units or features. Thus, the exemplary term "below" can encompass both above and below orientations. The device may be oriented in other ways (rotated 90 degrees or in other orientations), and the spatially relative descriptors used herein interpreted accordingly.
[0019] Embodiment 1 of the present invention provides an optimization method for a new energy system, wherein the new energy system is provided with a photovoltaic panel, a pumped storage device and a load;
[0020] like Figure 1 As shown, the following steps are included:
[0021] Step 101: creating a scheduling model corresponding to the new energy system, wherein the scheduling model is used to characterize how to perform power scheduling between photovoltaic panels and pumped storage equipment, so that the new energy system outputs stable electric energy;
[0022] Step 102: Model the scheduling model into a Markov decision process MDP, design its state space, action space and reward function; and use the DDPG algorithm to obtain an optimization strategy for the Markov decision process MDP.
[0023] The optimization method in the embodiment of the present invention can realize an intelligent scheduling method for efficient complementarity between photovoltaic power generation and pumped storage. By using a deep reinforcement learning algorithm to optimize the operation strategy of the photovoltaic-pumped storage system, the power fluctuation at the grid connection point is reduced, the economy and stability of the system operation are improved, 100% of photovoltaic power generation is achieved, and the operating efficiency of the energy storage equipment is optimized.
[0024] In this embodiment, in the Markov decision process MDP, the state s at time t t At least including: the output power P of the photovoltaic panel PV,t , the water level SOC of the pumped storage equipment t and the load power L t ; In state s t The action taken next t ; Reward function r(s t ,a t )=-α·(ΔP t ) 2 +β·ΔE tt , where ΔP t is the power change, ΔE t is the change in economic benefits, α and β are weight coefficients. Reward function r(s t ,a t ) is in state s t Take action a t The instant reward obtained, in the embodiment of the present invention, the reward function is based on economic benefits.
[0025] In this embodiment, action a t At least including: the pumping speed of the pumped storage equipment or the power generation power P of the photovoltaic panel PHS,t .
[0026] Here, when the pumping speed > 0, it means that water is pumped into the reservoir; otherwise, water is pumped out of the reservoir.
[0027] In this embodiment, the components of the DDPG algorithm include: an Actor network μ(s|θ) for mapping a given state s to an action a μ ), a critic network Q(s,a|θ) for evaluating the value of taking action a in state s Q ), target Actor network μ′(s|θ μ′ ) and the target Critic network Q′(s,a|θQ′ ); the critic network Q(s,a|θ Q ) is updated based on minimizing the mean square error between the predicted Q value and the target Q value; the Actor network μ(s|θ μ ) is updated based on the policy gradient method.
[0028] Here, the Critic network Q(s,a|θ Q ) can be specifically updated as follows: Among them, y i =r i +γQ′(s i+1 ,μ′(s i+1 ∣θ μ′ )|θ Q′ ), r i is the reward, γ is the discount factor;
[0029] The Actor network μ(s|θ μ ) can be specifically updated as follows: the gradient is
[0030] In this embodiment, the learning rate adjustment formula of the DDPG algorithm is: t =η 0 ×d (floor(t / T)) , where η 0 is the initial learning rate, d is the decay factor, t is the current number of iterations, T is the length of each cycle, and the floor function represents rounding down.
[0031] Here, the learning rate is not fixed, but decays with the training cycle. The purpose of this is to quickly explore in the early stage of training and fine-tune in the later stage to improve the stability and efficiency of learning.
[0032] In the inventor's practice, the following experiments were conducted:
[0033] Assumed initial conditions and parameters
[0034] 1. Photovoltaic output power P PV : Assume that the power output at time t is 50MW.
[0035] 2. State SOC of pumped storage equipment: initial water level is 50%.
[0036] 3. System load L: Assumed to be 45MW.
[0037] 4. Reward function weights: α=0.1, β=1.
[0038] 5. Discount factor γ: assumed to be 0.95.
[0039] Simulation of actions and results
[0040] 1. Assume action a t This is to increase power generation or pumping by 5MW, depending on the needs and status of the system.
[0041] 2. The new state caused by this action is SOC t+1 =50%-5%=45% (assuming water level consumed for power generation).
[0042] Reward Calculation
[0043] Assuming that after this action, the power fluctuation is reduced (from 10MW to 5MW), then:
[0044] 1. Power fluctuation ΔP t =5MW (reduced by 5MW).
[0045] 2. Change in economic benefits ΔE t =0.05 (assuming that every 1MW reduction in power fluctuation brings an increase of 0.01 in economic benefits).
[0046] The reward function is: r(s t ,a t )=-0.1·(5) 2 +1 0.05 = -2.5 + 0.05 = -2.45
[0047] State update and Q value calculation
[0048] Use the Critic network to calculate the Q value. Assuming that the parameters of the Q network are initialized with random values, the update of the Q value is as follows: t =r t +γQ′(s t+1 ,μ′(s t+1 ∣θ μ′ )|θ Q′ ), assuming Q′(s t+1 ,μ′(s t+1 ) is initially 3, then: t =-2.45+0.95×3=0.4.
[0049] Training of Actor and Critic Networks
[0050] 1. Critic update: minimize (y t -Q(s t ,a t ∣θ Q )) 2 , adjust θ by gradient descent Q .
[0051] 2. Actor Update: Through Policy Gradient Adjust θ μ .
[0052] Embodiment 2 of the present invention provides an optimization device for a new energy system, wherein the new energy system is provided with a photovoltaic panel, a pumped storage device and a load; and comprises the following modules:
[0053] A model creation module, used to create a scheduling model corresponding to the new energy system, wherein the scheduling model is used to characterize how to perform power scheduling between photovoltaic panels and pumped storage equipment, so that the new energy system outputs stable electric energy;
[0054] The optimization module is used to model the scheduling model into a Markov decision process MDP, design its state space, action space and reward function; and use the DDPG algorithm to obtain the optimization strategy of the Markov decision process MDP.
[0055] In this embodiment, in the Markov decision process MDP, the state s at time t t At least including: the output power P of the photovoltaic panel PV,t , the water level SOC of the pumped storage equipment t and the load power L t ; In state s t The action taken next t ; Reward function r(s t ,a t )=-α·(ΔP t ) 2 +β·ΔE t , where ΔP t is the power change, ΔE t is the change in economic benefits, and α and β are weight coefficients.
[0056] In this embodiment, action a t At least including: the pumping speed of the pumped storage equipment or the power generation power P of the photovoltaic panel PHS,t .
[0057] In this embodiment, the components of the DDPG algorithm include: an Actor network μ(s|θ) for mapping a given state s to an action a μ ), a critic network Q(s,a|θ) for evaluating the value of taking action a in state s Q ), target Actor network μ′(s|θ μ′ ) and the target Critic network Q′(s,a|θ Q′ );
[0058] The Critic network Q(s,a|θ Q ) is updated based on minimizing the mean square error between the predicted Q value and the target Q value;
[0059] The Actor network μ(s|θ μ ) is updated based on the policy gradient method.
[0060] In this embodiment, the learning rate adjustment formula of the DDPG algorithm is: t =η 0 ×d (floor(t / T)) , where η 0 is the initial learning rate, d is the decay factor, t is the current number of iterations, T is the length of each cycle, and the floor function represents rounding down.
[0061] It should be understood that although this specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation mode may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.
[0062] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.
Claims
1. An optimization method for a new energy system, wherein the new energy system is provided with a photovoltaic panel, a pumped storage device and a load; characterized in that: The following steps are involved: Creating a scheduling model corresponding to the new energy system, wherein the scheduling model is used to characterize how to perform power scheduling between photovoltaic panels and pumped storage equipment, so that the new energy system outputs stable electric energy; The scheduling model is modeled as a Markov decision process MDP, and its state space, action space and reward function are designed; the DDPG algorithm is used to obtain the optimization strategy of the Markov decision process MDP.
2. The optimization method according to claim 1, characterized in that: In the Markov decision process MDP, the state s at time t t At least including: the output power P of the photovoltaic panel PV,t , the water level SOC of the pumped storage equipment t and the load power L t ; In state s t The action taken next t ; Reward function r(s t ,a t )=-α·(ΔP t ) 2 +β·ΔE t , where ΔP t is the power change, ΔE t is the change in economic benefits, and α and β are weight coefficients.
3. The optimization method according to claim 2, characterized in that: Action a t At least including: the pumping speed of the pumped storage equipment or the power generation power P of the photovoltaic panel PHS,t .
4. The optimization method according to claim 3, characterized in that: The components of the DDPG algorithm include: an Actor network μ(s|θ) for mapping a given state s to an action a μ ), a critic network Q(s,a|θ) for evaluating the value of taking action a in state s Q ), target Actor network μ′(s|θ μ′ ) and the target Critic network Q′(s,a|θ Q′ ); The Critic network Q(s,a|θ Q ) is updated based on minimizing the mean square error between the predicted Q value and the target Q value; The Actor network μ(s|θ μ ) is updated based on the policy gradient method.
5. The optimization method according to claim 4, characterized in that: The learning rate adjustment formula of the DDPG algorithm is: t =η0×d (floor(t / T)) , where η0 is the initial learning rate, d is the decay factor, t is the current iteration number, T is the length of each cycle, and the floor function represents rounding down.
6. An optimization device for a new energy system, wherein the new energy system is provided with a photovoltaic panel, a pumped storage device and a load; characterized in that: Includes the following modules: A model creation module, used to create a scheduling model corresponding to the new energy system, wherein the scheduling model is used to characterize how to perform power scheduling between photovoltaic panels and pumped storage equipment, so that the new energy system outputs stable electric energy; The optimization module is used to model the scheduling model into a Markov decision process MDP, design its state space, action space and reward function; and use the DDPG algorithm to obtain the optimization strategy of the Markov decision process MDP.
7. The optimization device according to claim 6, characterized in that: In the Markov decision process MDP, the state s at time t t At least including: the output power P of the photovoltaic panel PB,t , the water level SOC of the pumped storage equipment t and the load power L t ; In state s t The action taken next t ; Reward function r(s t ,a t )=-α·(ΔP t ) 2 +β·ΔE t , where ΔP t is the power change, ΔE t is the change in economic benefits, and α and β are weight coefficients.
8. The optimization device according to claim 7, characterized in that: Action a t At least including: the pumping speed of the pumped storage equipment or the power generation power P of the photovoltaic panel PHS,t .
9. The optimization device according to claim 8, characterized in that: The components of the DDPG algorithm include: an Actor network μ(s|θ) for mapping a given state s to an action a μ ), a critic network Q(s,a|θ) for evaluating the value of taking action a in state s Q ), target Actor network μ′(s|θ μ′ ) and the target Critic network Q′(s,a|θ Q′ ); The Critic network Q(s,a|θ Q ) is updated based on minimizing the mean square error between the predicted Q value and the target Q value; The Actor network μ(s|θ μ ) is updated based on the policy gradient method.
10. The optimization device according to claim 9, characterized in that: The learning rate adjustment formula of the DDPG algorithm is: t =η0×d (floor(t / T)) , where η0 is the initial learning rate, d is the decay factor, t is the current iteration number, T is the length of each cycle, and the floor function represents rounding down.