Storage medium, multi-energy coupling system control method, device and equipment

Abstract

The invention discloses a multi-energy coupling system control method, device and equipment and a storage medium, and the method comprises the steps: constructing a multi-energy coupling model which comprises a wind power generation sub-model, a photovoltaic power generation sub-model, a battery energy storage sub-model and a water electrolysis hydrogen production sub-model for a wind-light-electricity hydrogen storage multi-energy coupling system; acquiring current state information of the wind-light-electricity hydrogen storage multi-energy coupling system; the current state information is used for the multi-energy coupling model; an action instruction is generated according to the multi-energy coupling model by adopting a reinforcement learning algorithm of an epsilon-greedy strategy; the action instruction is an action instruction capable of enabling the state information reward value of the multi-energy coupling model at the next time step length to be the highest in an experience pool of the reinforcement learning algorithm; a value function in the reinforcement learning algorithm comprises a preset penalty function; according to the method, when the reward value is calculated for the executed action instruction, the specific penalty function is introduced, the economical efficiency can be maximized while the stability of the multi-energy coupling system is ensured, and then the optimal action instruction is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-energy coupling systems, and in particular to a storage medium, a multi-energy coupling system control method, a device and equipment. Background Art

[0002] In recent years, local consumption technology based on renewable energy power generation has become a hot topic in the new energy field. Multi-energy coupling systems involving wind energy, solar energy, electric energy storage and hydrogen energy have become typical application solutions for local consumption. It not only solves the local consumption of renewable energy electricity, but also converts electricity into a new type of green energy, hydrogen.

[0003] The multi-energy coupling system involves the mutual influence and constraints among multiple energy sources such as wind energy, solar energy, electric energy storage, and hydrogen energy. Fluctuations in any link will lead to the collapse of the entire system.

[0004] The inventors have found through research that the multi-energy coupling system in the prior art still has at least the following defects:

[0005] Due to the large number and types of equipment involved, the mutual influence and constraints between the equipment will result in poor control effect under conventional control methods, resulting in poor stability and economy of the multi-energy coupling system.

[0006] The information disclosed in this background technology section is only intended to enhance the understanding of the overall background of the invention and should not be regarded as an acknowledgment or any form of suggestion that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0007] The purpose of the present invention is to improve the stability and economy of the multi-energy coupling system.

[0008] The present invention provides a multi-energy coupling system control method, comprising the steps of:

[0009] S11. For the wind, photovoltaic and hydrogen storage multi-energy coupling system, a multi-energy coupling model including wind power generation electronic model, photovoltaic generation electronic model, battery energy storage sub-model and water electrolysis hydrogen production sub-model is constructed;

[0010] S12, obtaining the current state information S of the wind-solar-electricity-hydrogen-storage multi-energy coupling system (t) The current status information S (t) For the multi-energy coupling model;

[0011] S13, generating action instructions A according to the multi-energy coupling model by using a reinforcement learning algorithm adopting the ε-greedy strategy (t) ; The action instruction A (t) is the state information S in the experience pool of the reinforcement learning algorithm that enables the multi-energy coupling model to be in the next time step(t+1) The action instruction with the highest reward value; the value function in the reinforcement learning algorithm includes a preset penalty function; the penalty function includes:

[0012]

[0013] In formula (1), is the capacity of the hydrogen storage tank; is the energy storage battery capacity; Q is the hydrogen storage value of the hydrogen storage tank at time t+1; bat (t+1) is the charge of the energy storage battery at time t+1.

[0014] Preferably, the present invention further comprises:

[0015] According to the state information S (t+1) Calculate the corresponding reward value R (t+1) And generate a new value information record;

[0016] Updating the experience pool according to the new value information record includes:

[0017] Q new (S t ,A t )←Q old (S t ,A t )+α(R t +γ·Q(S t+1 ,A*)-Q old (S t ,A t )), formula (2);

[0018] In formula (2), Q new (S t ,A t ) is the updated state information S at time t t and action A t The value information under Q old (S t ,A t ) is the state information S at time t before updating t and action A t The value information under R t is the reward value, A * is the state information S at time t+1 t+1 The best historical action performed under the condition; α is the autonomous learning rate; γ is the reward decay rate.

[0019] In another aspect of the present invention, a multi-energy coupling system control device is also provided, comprising:

[0020] Modeling unit, used to construct a multi-energy coupling model including wind power generation sub-model, photovoltaic power generation sub-model, battery energy storage sub-model and water electrolysis hydrogen production sub-model for the wind, photovoltaic and hydrogen storage multi-energy coupling system; (modeling unit)

[0021] An information collection unit is used to obtain the current state information S of the wind-solar-electricity-hydrogen-storage multi-energy coupling system. (t) The current status information S (t) For the multi-energy coupling model;

[0022] An action instruction generating unit is used to generate an action instruction A according to the multi-energy coupling model by using a reinforcement learning algorithm adopting an ε-greedy strategy. (t) ; The action instruction A (t) is the state information S in the experience pool of the reinforcement learning algorithm that enables the multi-energy coupling model to be in the next time step (t+1) The action instruction with the highest reward value; the value function in the reinforcement learning algorithm includes a preset penalty function; the penalty function includes:

[0023]

[0024] In formula (1), is the capacity of the hydrogen storage tank; is the energy storage battery capacity; Q is the hydrogen storage value of the hydrogen storage tank at time t+1; bat (t+1) is the charge of the energy storage battery at time t+1.

[0025] Preferably, the present invention further comprises:

[0026] An experience pool updating unit is used to update the state information S (t+1) Calculate the corresponding reward value R (t+1) and generating a new value information record; and updating the experience pool according to the new value information record, including:

[0027] Q new (S t ,A t )←Q old (S t ,A t )+α(R t +γ·Q(S t+1 ,A*)-Q old (S t ,A t )), formula (2);

[0028] In formula (2), Q new (St ,A t ) is the updated state information S at time t t and action A t The value information under Q old (S t ,A t ) is the state information S at time t before updating t and action A t The value information under R t is the reward value, A * is the state information S at time t+1 t+1 The best historical action performed under the condition; α is the autonomous learning rate; γ is the reward decay rate.

[0029] In another aspect of the embodiment of the present invention, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, each step of the multi-energy coupling system control method as described in any one of the above items is implemented.

[0030] On the other hand, a multi-energy coupling system control device is provided; the multi-energy coupling system control device includes a computer program stored on a medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the methods described in the above aspects and achieves the same technical effects.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] In the present invention, a multi-energy coupling model including a wind power generation sub-model, a photovoltaic power generation sub-model, a battery energy storage sub-model and a water electrolysis hydrogen production sub-model is first constructed to simulate a real multi-energy coupling system; then, the state information of the multi-energy coupling system is acquired through an information acquisition module to provide reliable data support for calculating the reward value through the multi-energy coupling model; when calculating the reward value for the executed action instruction, the present invention introduces a specific penalty function to ensure the stability of the multi-energy coupling system while maximizing the economy of the multi-energy coupling system, so that the optimal action instruction can be selected for the operation of the multi-energy coupling system at the next moment.

[0033] In the present invention, according to the solution results of each reward value, the value information including state information, action instructions and reward value information is recorded and stored in the experience pool, and the action instructions with the best historical value information in the experience pool are input into the multi-energy coupling model for autonomous learning; in this way, the experience pool is continuously updated through autonomous learning, and the optimal operation strategy of the multi-energy coupling system is determined, which can further improve the stability of the multi-energy coupling system.

[0034] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, and to make the above and other purposes, technical features and advantages of the present invention easier to understand, one or more preferred embodiments are listed below and described in detail with the accompanying drawings as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 is a step diagram of the multi-energy coupling system control method of the present invention;

[0037] Figure 2 It is a structural schematic diagram of the multi-energy coupling system control device of the present invention;

[0038] Figure 3 It is a schematic diagram of the structure of the multi-energy coupling system control device described in the present invention. DETAILED DESCRIPTION

[0039] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific implementation modes.

[0040] Unless explicitly stated otherwise, throughout the specification and claims, the term “comprise” or variations such as “include” or “comprising”, etc., will be understood to include the stated elements or components but not to exclude other elements or components.

[0041] In this document, the terms "first", "second", etc. are used to distinguish two different elements or parts, and are not used to limit a specific position or relative relationship. In other words, in some embodiments, the terms "first", "second", etc. can also be interchangeable.

[0042] Embodiment 1

[0043] In order to improve the stability and economy of multi-energy coupling systems, Figure 1 As shown, in an embodiment of the present invention, a multi-energy coupling system control method is provided, comprising the steps of:

[0044] S11. For the wind, photovoltaic and hydrogen storage multi-energy coupling system, a multi-energy coupling model including wind power generation electronic model, photovoltaic generation electronic model, battery energy storage sub-model and water electrolysis hydrogen production sub-model is constructed;

[0045] In order to realize the local consumption of various new energy sources mainly through the coordinated application of wind energy, solar energy, electric energy storage, hydrogen energy and other energy sources, the wind, photovoltaic, hydrogen storage and multi-energy coupling system in the embodiment of the present invention includes several important core devices such as wind power generation unit, photovoltaic power generation unit, energy storage battery unit and water electrolysis hydrogen production unit, as well as an AC bus for connecting to the external power grid for electric energy exchange and an intelligent control unit for realizing coordinated control of each unit.

[0046] The intelligent control unit in the embodiment of the present invention can be a computing device with data processing capabilities, that is, by calling and executing a set computer program to generate action instructions for the wind, photovoltaic, hydrogen storage multi-energy coupling system. Specifically, through various forms of state detection, it is possible to obtain monitoring data corresponding to various state information of each unit in the wind, photovoltaic, hydrogen storage multi-energy coupling system in real time. The state information includes but is not limited to energy state information; such as wind power output power, photovoltaic power output power, exchange power between AC bus and power grid, energy storage battery charge state, hydrogen storage tank hydrogen demand load and other data.

[0047] The multi-energy coupling model in the embodiment of the present invention is constructed based on the actual wind, photovoltaic and hydrogen storage multi-energy coupling system, and can be specifically composed of several models, including a wind power generation sub-model, a photovoltaic power generation sub-model, a battery energy storage sub-model and a water electrolysis hydrogen production sub-model.

[0048] In practical applications, the multi-energy coupling model in the embodiment of the present invention can be specifically as follows:

[0049]

[0050] In formula (3), P pv(t) , P wt(t) , P H2(t) and E bat(t) are the photovoltaic output power, wind turbine output power, electrolyzer power consumption and energy storage battery charge state within time t; η pv(t) , η wt(t) , η H2(t) and η bat They are the photovoltaic power generation efficiency, wind power generation efficiency, electrolysis efficiency of the electrolyzer, and charging and discharging efficiency of the energy storage battery within time t; and are the maximum output power of photovoltaic power generation, the maximum output power of wind power generation and the maximum electrolysis power of electrolyzer; P bat(t) , Q bat are the battery charging and discharging power and the capacity of the energy storage battery in time t respectively;

[0051] The operating condition constraints that each model in the multi-energy coupling model should meet include:

[0052]

[0053] In formula (4), and They are photovoltaic power generation, wind power generation, electrolyzer power consumption and lower limit of charge and discharge power; and They are photovoltaic power generation, wind power generation, electrolyzer power consumption and upper limit of charge and discharge power;

[0054] The power balance constraints that the multi-energy coupling model needs to satisfy include:

[0055]

[0056] In formula (5), P pv(t) , P wt(t) , P bat(t) and P H2(t) are the photovoltaic power generation, wind power generation, electrolyzer charging and discharging power, and electrolyzer power consumption at time t respectively; P pg(t) is the exchange power between the multi-energy coupling system and the grid through the AC bus at time t.

[0057] S12, obtaining the current state information S of the wind-solar-electricity-hydrogen-storage multi-energy coupling system (t) ; The current state information S (t) For the multi-energy coupling model;

[0058] The state information S is generated by the monitoring data obtained by the information collection modules at multiple key nodes in each distributed energy system of the wind-photovoltaic-hydrogen storage multi-energy coupling system. (t) ; The state information S (t) include:

[0059]

[0060] In formula (6), P pv(t) is the photovoltaic output power at time t; P wt(t) is the wind turbine output power at time t; Q H2(t) is the hydrogen storage capacity of the hydrogen storage tank at time t; L H2(t) is the hydrogen demand side load at time t.

[0061] S13, generating action instructions A according to the multi-energy coupling model by using a reinforcement learning algorithm adopting the ε-greedy strategy (t) ; The action instruction A (t) is the state information S in the experience pool of the reinforcement learning algorithm that enables the multi-energy coupling model to be in the next time step (t+1)The action instruction with the highest reward value; the value function in the reinforcement learning algorithm includes a preset penalty function; the penalty function includes:

[0062]

[0063] In formula (1), is the capacity of the hydrogen storage tank; is the capacity of the energy storage battery; is the hydrogen storage value of the hydrogen storage tank at time t + 1; Q bat (t + 1) is the charge of the energy storage battery at time t + 1.

[0064] The reinforcement learning algorithm based on the ε-greedy strategy in the embodiments of the present invention; the action instruction A (t) refers to the action instruction that can make the multi-energy coupling model have the highest reward value in the state information S at the next time step in the experience pool of the reinforcement learning algorithm, and specifically may include: (t+1) In formula (7), P

[0065]

[0066] is the photovoltaic output power at time t; P pv(t) is the output power of the wind turbine at time t; P wt(t) is the power consumption of the electrolyzer at time t; H2(t) It should be noted that in the embodiments of the present invention, the content recorded in the value information in the experience pool may specifically be the state information, action execution information, and reward value information corresponding to a moment t.

[0067]

[0068] Generating the action instruction A (t) based on the multi-energy coupling model, specifically may include:

[0069] The ε-greedy strategy includes:

[0070]

[0071] In formula (8), O(A t 丨S t ) is the strategy of making a decision to execute the action A t in the state S at time t, ε is the greediness coefficient of the ε-greedy strategy, and R(s) is the reward value of executing the action A t in the state S t at time t. t

[0072] Furthermore, in the embodiments of the present invention, generating the action instruction A(t) based on the multi-energy coupling model may further include:

[0073] Generate the state information S at the next moment (t+1) according to the multi-energy coupling model (t+1) :

[0074]

[0075] In formula (9), P pv(t+1) , P wt(t+1) , P bat(t+1) and P H2(t+1) are the photovoltaic power generation power, wind power generation power, electrolyzer charge and discharge power, and electrolyzer power consumption at time t+1 respectively; P pg(t+1) is the exchange power between the multi-energy coupling system and the power grid through the AC bus at time t+1.

[0076] In the embodiment of the present invention, in order to reduce the overall storage cost by reducing the storage cost of the energy storage battery and / or the hydrogen storage cost of the hydrogen tank, the value function in the chemical learning algorithm of the embodiment of the present invention also introduces a penalty function f (t) , that is, formula (1); it can be seen from the value function that when the reward function benefit value is the largest and the penalty function penalty value is the smallest, the reward value is the largest. At this time, the electric energy generated by the wind-solar-hydrogen energy storage multi-energy coupling system is transmitted to the electrolyzer for hydrogen production and sold to the power grid. There is no need for the energy storage battery to store electricity, and the hydrogen produced by the electrolyzer is directly transmitted to the hydrogen load without storing hydrogen in the hydrogen storage tank, reducing the penalty value to the lowest. At this time, the wind-solar-hydrogen energy storage multi-energy coupling system will work in the optimal economic state.

[0077] In practical applications, the value function in the reinforcement learning algorithm in the embodiment of the present invention may specifically include:

[0078] R t (S t |A t ) = p h *L H2 (t+1) + p e *P pg (t+1) - f(t), formula (10);

[0079] In formula (10), R t (S t 丨A t ) is the reward value obtained by executing the action strategy A t in the state of S at time t; p t h is the unit price of selling hydrogen produced by electrolysis to the hydrogen refueling station; p h (t) is the unit price of selling electric energy from the multi-energy coupling system to the power grid through the AC bus; f (t) is the penalty function.

[0080] Preferably, in order to obtain better control effects through continuous autonomous learning, the embodiment of the present invention may further include the following steps of updating the experience pool:

[0081] According to the status information S (t+1) Calculate the corresponding reward value R (t+1) And generate a new value information record;

[0082] Update the experience pool based on the new value information record, including:

[0083] Q new (S t ,A t )←Q old (S t ,A t )+α(R t +γ·Q(S t+1 ,A*)-Q old (S t ,A t )), formula (2);

[0084] In formula (2), Q new (S t ,A t ) is the updated state information S at time t t and action A t The value information under Q old (S t ,A t ) is the state information S at time t before updating t and action A t The value information under R t is the reward value, A * is the state information S at time t+1 t+1 The best historical action performed under the condition; α is the autonomous learning rate; γ is the reward decay rate.

[0085] In summary, in an embodiment of the present invention, a multi-energy coupling model including a wind power generation sub-model, a photovoltaic power generation sub-model, a battery energy storage sub-model and a water electrolysis hydrogen production sub-model is first constructed to simulate a real multi-energy coupling system; then, the state information of the multi-energy coupling system is acquired through an information acquisition module to provide reliable data support for calculating the reward value through the multi-energy coupling model; when calculating the reward value by calculating the executed action instruction, an embodiment of the present invention introduces a specific penalty function to ensure the stability of the multi-energy coupling system while maximizing the economy of the multi-energy coupling system, so that the optimal action instruction can be selected for the operation of the multi-energy coupling system at the next moment.

[0086] In an embodiment of the present invention, value information records including state information, action instructions and reward value information are stored in an experience pool based on the solution results of each reward value, and the action instructions with the best historical value information in the experience pool are input into the multi-energy coupling model for autonomous learning; in this way, the experience pool is continuously updated through autonomous learning, and the optimal operation strategy of the multi-energy coupling system is determined, which can further improve the stability of the multi-energy coupling system.

[0087] Embodiment 2

[0088] Corresponding to the method embodiment, another aspect of the embodiment of the present invention further provides a multi-energy coupling system control device. Figure 2 The schematic diagram of the structure of the multi-energy coupling system control device provided by the embodiment of the present invention is shown, and the residual failure rate calculation device is Figure 1 The device corresponding to the multi-energy coupling system control method described in the corresponding embodiment, that is, implemented by means of a virtual device Figure 1 In the corresponding embodiment of the multi-energy coupling system control method, each virtual module constituting the multi-energy coupling system control device can be executed by an electronic device, such as a network device, a terminal device or a server. Specifically, the multi-energy coupling system control device in the embodiment of the present invention includes:

[0089] Modeling unit 01 is used to construct a multi-energy coupling model including a wind power generation sub-model, a photovoltaic generation sub-model, a battery energy storage sub-model and a water electrolysis hydrogen production sub-model for the wind, photovoltaic and hydrogen storage multi-energy coupling system; (modeling unit)

[0090] Information collection unit 02, used to obtain the current state information S of the wind-solar-electricity-hydrogen-storage multi-energy coupling system (t) ; The current state information S (t) For the multi-energy coupling model;

[0091] The action instruction generating unit 03 is used to generate an action instruction A according to the multi-energy coupling model by using a reinforcement learning algorithm adopting an ε-greedy strategy. (t) ; The action instruction A (t) is the state information S in the experience pool of the reinforcement learning algorithm that enables the multi-energy coupling model to be in the next time step (t+1) The action instruction with the highest reward value; the value function in the reinforcement learning algorithm includes a preset penalty function; the penalty function includes:

[0092]

[0093] In formula (1), is the capacity of the hydrogen storage tank; is the energy storage battery capacity; 1) is the hydrogen storage value of the hydrogen storage tank at time t+1; Q bat (t+1) is the charge of the energy storage battery at time t+1.

[0094] Preferably, the present invention further comprises:

[0095] An experience pool updating unit (not shown in the figure) is used to update the state information S (t+1) Calculate the corresponding reward value R (t+1) and generating a new value information record; and updating the experience pool according to the new value information record, including:

[0096] Q new (S t ,A t )←Q old (S t ,A t )+α(R t +γ·Q(S t+1 ,A*)-Q old (S t ,A t )), formula (2);

[0097] In formula (2), Q new (S t ,A t ) is the updated state information S at time t t and action A t The value information under Q old (S t ,A t ) is the state information S at time t before updating t and action A t The value information under R t is the reward value, A * is the state information S at time t+1 t+1 The best historical action performed under the condition; α is the autonomous learning rate; γ is the reward decay rate.

[0098] It should be noted that the specific implementation method and technical effects of the multi-energy coupling system control device in the embodiment of the present invention can be referred to Figure 1 The corresponding multi-energy coupling system control method will not be described in detail here.

[0099] Embodiment 3

[0100] Corresponding to the method embodiment, in the embodiment of the present invention, a multi-energy coupling system control device, such as a terminal, a server, etc., is also provided. Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto.

[0101] An example of a hardware structure block diagram of a multi-energy coupling system control device provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown, it may include:

[0102] Processor 1, communication interface 2, memory 3 and communication bus 4;

[0103] The processor 1, the communication interface 2, and the memory 3 communicate with each other via the communication bus 4;

[0104] Optionally, the communication interface 2 may be an interface of a communication module, such as an interface of a GSM module;

[0105] The processor 1 may be a central processing unit CPU, or an application-specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application.

[0106] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0107] The processor 1 is specifically used to execute the computer program stored in the memory 3 to perform the following steps:

[0108] S11. For the wind, photovoltaic and hydrogen storage multi-energy coupling system, a multi-energy coupling model including wind power generation electronic model, photovoltaic generation electronic model, battery energy storage sub-model and water electrolysis hydrogen production sub-model is constructed;

[0109] S12, obtaining the current state information S of the wind-solar-electricity-hydrogen-storage multi-energy coupling system (t) ; The current state information S (t) For the multi-energy coupling model;

[0110] S13, generating action instructions A according to the multi-energy coupling model by using a reinforcement learning algorithm adopting the ε-greedy strategy (t); The action instruction A (t) is the state information S in the experience pool of the reinforcement learning algorithm that enables the multi-energy coupling model to be in the next time step (t+1) The action instruction with the highest reward value; the value function in the reinforcement learning algorithm includes a preset penalty function; the penalty function includes:

[0111]

[0112] In formula (1), is the capacity of the hydrogen storage tank; is the energy storage battery capacity; Q is the hydrogen storage value of the hydrogen storage tank at time t+1; bat (t+1) is the charge of the energy storage battery at time t+1.

[0113] Preferably, the present invention further comprises:

[0114] According to the state information S (t+1) Calculate the corresponding reward value R (t+1) And generate a new value information record;

[0115] Updating the experience pool according to the new value information record includes:

[0116] Q new (S t ,A t )←Q old (S t ,A t )+α(R t +γ·Q(S t+1 ,A*)-Q old (S t ,A t )), formula (2);

[0117] In formula (2), Q new (S t ,A t ) is the updated state information S at time t t and action A t The value information under Q old (S t ,A t ) is the state information S at time t before updating t and action A t The value information under R t is the reward value, A * is the state information S at time t+1 t+1 The best historical action performed under the condition; α is the autonomous learning rate; γ is the reward decay rate.

[0118] The above product can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the multi-energy coupling system control method provided by the embodiment of the present invention.

[0119] Embodiment 4

[0120] In an embodiment of the present invention, a storage medium is further provided, which may store a program suitable for execution by a processor, wherein the program is used to:

[0121] S11. For the wind, photovoltaic and hydrogen storage multi-energy coupling system, a multi-energy coupling model including wind power generation electronic model, photovoltaic generation electronic model, battery energy storage sub-model and water electrolysis hydrogen production sub-model is constructed;

[0122] S12, obtaining the current state information S of the wind-solar-electricity-hydrogen-storage multi-energy coupling system (t) ; The current state information S (t) For the multi-energy coupling model;

[0123] S13, generating action instructions A according to the multi-energy coupling model by using a reinforcement learning algorithm adopting the ε-greedy strategy (t) ; The action instruction A (t) is the state information S in the experience pool of the reinforcement learning algorithm that enables the multi-energy coupling model to be in the next time step (t+1) The action instruction with the highest reward value; the value function in the reinforcement learning algorithm includes a preset penalty function; the penalty function includes:

[0124]

[0125] In formula (1), is the capacity of the hydrogen storage tank; is the energy storage battery capacity; Q is the hydrogen storage value of the hydrogen storage tank at time t+1; bat (t+1) is the charge of the energy storage battery at time t+1.

[0126] Preferably, the present invention further comprises:

[0127] According to the state information S (t+1) Calculate the corresponding reward value R (t+1) And generate a new value information record;

[0128] Updating the experience pool according to the new value information record includes:

[0129] Q new (St ,A t )←Q old (S t ,A t )+α(R t +γ·Q(S t+1 ,A*)-Q old (S t ,A t )), formula (2);

[0130] In formula (2), Q new (S t ,A t ) is the updated state information S at time t t and action A t The value information under Q old (S t ,A t ) is the state information S at time t before updating t and action A t The value information under R t is the reward value, A * is the state information S at time t+1 t+1 The best historical action performed under the condition; α is the autonomous learning rate; γ is the reward decay rate.

[0131] Optionally, the detailed functions and extended functions of the program may refer to the above description.

[0132] The above product can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to the methods provided by other embodiments of the present invention.

[0133] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0134] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0135] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0137] It should be understood that in the embodiments of the present application, the various embodiments and features can be combined with each other to solve the aforementioned technical problems.

[0138] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0139] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-energy coupling system control method, characterized in that: Includes steps: S11. For the wind, photovoltaic and hydrogen storage multi-energy coupling system, a multi-energy coupling model including wind power generation electronic model, photovoltaic generation electronic model, battery energy storage sub-model and water electrolysis hydrogen production sub-model is constructed; S12, obtaining the current state information S of the wind-solar-electricity-hydrogen-storage multi-energy coupling system (t) The current status information S (t) For the multi-energy coupling model; S13, generating action instructions A according to the multi-energy coupling model by using a reinforcement learning algorithm adopting the ε-greedy strategy (t) ; The action instruction A (t) is the state information S in the experience pool of the reinforcement learning algorithm that enables the multi-energy coupling model to be in the next time step (t+1) The action command with the highest reward value; The value function in the reinforcement learning algorithm includes a preset penalty function; the penalty function includes: In formula (1), is the capacity of the hydrogen storage tank; is the energy storage battery capacity; Q is the hydrogen storage value of the hydrogen storage tank at time t+1; bat (t+1) is the charge of the energy storage battery at time t+1.

2. The multi-energy coupling system control method according to claim 1, characterized in that: The content of the value information recorded in the experience pool includes: The state information, execution action information and reward value information corresponding to a moment t.

3. The multi-energy coupling system control method according to claim 2, characterized in that: Also includes: According to the state information S (t+1) Calculate the corresponding reward value R (t+1) And generate a new value information record; Updating the experience pool according to the new value information record includes: Q new (S t ,A t )←Q old (S t ,A t )+α(R t +γ·Q(S t+1 ,A*)-Q old (S t ,A t )),,in(2)! In formula (2), Q new (S t ,A t ) is the updated state information S at time t t and action A t The value information under Q old (S t ,A t ) is the state information S at time t before updating t and action A t The value information under R t is the reward value, A * is the state information S at time t+1 t+1 The best historical action performed under the condition; α is the autonomous learning rate; γ is the reward decay rate.

4. The multi-energy coupling system control method according to claim 3, characterized in that: The multi-energy coupling model includes: In formula (3), P pv(t) , P wt(t) , P H2(t) and E bat(t) are the photovoltaic output power, wind turbine output power, electrolyzer power consumption and energy storage battery charge state within time t; η pv(t) , η wt(t) , η H2(t) and η bat They are the photovoltaic power generation efficiency, wind power generation efficiency, electrolysis efficiency of the electrolyzer, and charging and discharging efficiency of the energy storage battery within time t; and are the maximum output power of photovoltaic power generation, the maximum output power of wind power generation and the maximum electrolysis power of electrolyzer; P bat(t) , Q bat are the battery charging and discharging power and the capacity of the energy storage battery in time t respectively; The operating condition constraints that each model in the multi-energy coupling model should meet include: In formula (4), and They are photovoltaic power generation, wind power generation, electrolyzer power consumption and lower limit of charge and discharge power; and They are photovoltaic power generation, wind power generation, electrolyzer power consumption and upper limit of charge and discharge power; The power balance constraints that the multi-energy coupling model needs to satisfy include: In formula (5), P pv(t) , P wt(t) , P bat(t) and P H2(t) are the photovoltaic power generation, wind power generation, electrolyzer charging and discharging power, and electrolyzer power consumption at time t respectively; P pg(t) is the exchange power between the multi-energy coupling system and the grid through the AC bus at time t.

5. The multi-energy coupling system control method according to claim 4, characterized in that: The current state information S of the wind-solar-electricity-hydrogen-storage multi-energy coupling system (t) ,include: The state information S is generated by the monitoring data obtained by the information collection modules at multiple key nodes in each distributed energy system of the wind-photovoltaic-hydrogen storage multi-energy coupling system. (t) ; The state information S (t) include: In formula (6), P pv(t) is the photovoltaic output power at time t; P wt(t) is the wind turbine output power at time t; is the hydrogen storage capacity of the hydrogen storage tank at time t; is the hydrogen demand side load at time t.

6. The multi-energy coupling system control method according to claim 5, characterized in that: The action instruction A (t) ,include: In formula (7), P pv(t) is the photovoltaic output power at time t; P wt(t) is the wind turbine output power at time t; is the power consumed by the electrolytic cell at time t.

7. The multi-energy coupling system control method according to claim 6, characterized in that: The reinforcement learning algorithm adopts the ε-greedy strategy to generate action instructions A according to the multi-energy coupling model. (t) ,include: The ε-greedy strategy includes: In formula (8), O(A t 丨S t ) is the policy for executing action A t in the state of S at time t, ε is the greediness coefficient of the ε-greedy policy, and R(s) is the reward value for executing action A t in the state of S at time t t when executing action A t .

8. The multi-energy coupling system control method according to claim 7, characterized in that: The reinforcement learning algorithm adopts the ε-greedy strategy to generate action instructions A according to the multi-energy coupling model. (t) , also includes: Generate the state information S at the next moment (t+1) according to the multi-energy coupling model (t+1) : In formula (9), P pv(t+1) , P wt(t+1) , P bat(t+1) and P H2(t+1) are the photovoltaic power generation, wind power generation, electrolyzer charging and discharging power, and electrolyzer power consumption at time t+1 respectively; P pg(t+1) It is the exchange power between the multi-energy coupling system and the grid through the AC bus at time t+1.

9. The multi-energy coupling system control method according to claim 8, characterized in that: The value function in the reinforcement learning algorithm includes: In formula (10), R t (S t 丨A t ) is the reward value obtained by executing the action policy A in the state of S at time t t ; p t is the unit price of selling the hydrogen produced by electrolysis for hydrogen refueling stations; p h is the unit price of selling electric energy from the multi - energy coupling system to the power grid through the AC bus; f h is the penalty function.​​ 10. A multi-energy coupling system control device, characterized in that: include: Modeling unit, used to construct a multi-energy coupling model including wind power generation sub-model, photovoltaic power generation sub-model, battery energy storage sub-model and water electrolysis hydrogen production sub-model for the wind, photovoltaic and hydrogen storage multi-energy coupling system; (modeling unit) An information collection unit is used to obtain the current state information S of the wind-solar-electricity-hydrogen-storage multi-energy coupling system. (t) ; The current state information S (t) For the multi-energy coupling model; An action instruction generating unit is used to generate an action instruction A according to the multi-energy coupling model by using a reinforcement learning algorithm adopting an ε-greedy strategy. (t) ; The action instruction A (t) is the state information S in the experience pool of the reinforcement learning algorithm that enables the multi-energy coupling model to be in the next time step (t+1) The action command with the highest reward value; The value function in the reinforcement learning algorithm includes a preset penalty function; the penalty function includes: In formula (1), is the capacity of the hydrogen storage tank; is the energy storage battery capacity; Q is the hydrogen storage value of the hydrogen storage tank at time t+1; bat (t+1) is the charge of the energy storage battery at time t+1.

11. The multi-energy coupling system control device according to claim 10, characterized in that: Also includes: An experience pool updating unit is used to update the state information S (t+1) Calculate the corresponding reward value R (t+1) And generate a new value information record; Updating the experience pool according to the new value information record includes: Q new (S t ,A t )←Q old (S t ,A t )+α(R t +γ·Q(S t+1 ,A*)-Q old (S t ,A t )),,in(2)! In formula (2), Q new (S t ,A t ) is the updated state information S at time t t and action A t The value information under Q old (S t ,A t ) is the state information S at time t before updating t and action A t The value information under R t is the reward value, A * is the state information S at time t+1 t+1 The best historical action performed under the condition; α is the autonomous learning rate; γ is the reward decay rate.

12. A multi-energy coupling system control device, characterized in that: include: Memory for storing computer programs; A processor is used to call and execute the computer program to implement the steps of the multi-energy coupling system control method as described in any one of claims 1 to 9.

13. A storage medium, characterized in that: The method comprises a software program, wherein the software program is suitable for a processor to execute the steps of the multi-energy coupling system control method as claimed in any one of claims 1 to 9.

Images