Industrial park energy storage standby power supply control method based on improved deep reinforcement learning

Abstract

The invention discloses an industrial park energy storage standby power supply control method based on improved deep reinforcement learning, and relates to the technical field of energy management. Comprising the following steps: S1, acquiring energy use data of an industrial park in real time through a source / network / load / storage model, the energy use data comprising state information of a power supply, a power grid, a load and energy storage equipment; s2, preprocessing the energy use data; and S3, judging whether the external power supply of the industrial park breaks down or not, and if so, substituting the preprocessed energy use data into the deep reinforcement learning model to generate an action data set of the energy storage power supply. The system can effectively manage and control the industrial park energy storage standby power supply, and has good economic benefits and practical values.

Description

Technical Field

[0001] The present invention relates to the field of energy management technology, and in particular to an industrial park energy storage backup power supply control method based on improved deep reinforcement learning. Background Art

[0002] As environmental pollution worsens, the State Grid has attached more and more importance to new energy. At present, the more mature new energy technologies are wind and solar power generation. However, due to the instability of these two energy sources, energy storage backup power has become an important link in achieving stable and reliable power generation from new energy. A high-performance energy storage control and management system can ensure the power supply quality and reliability of the new energy system. The high energy ratio characteristics of lithium batteries have safety hazards, and incorrect use will affect the safety of battery use. Therefore, battery charging and discharging must be strictly managed according to battery parameters. In addition, factors such as ambient temperature, humidity, and depth of discharge will affect the service life of the battery. It is difficult to obtain accurate estimation results for lithium battery status using general methods. Summary of the invention

[0003] In order to solve at least one of the technical problems mentioned in the background technology, the purpose of the present invention is to provide an industrial park energy storage backup power supply control method based on improved deep reinforcement learning, which can effectively manage and control the industrial park energy storage backup power supply and has good economic benefits and practical value.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] A method for controlling an industrial park energy storage backup power supply based on improved deep reinforcement learning comprises the following steps:

[0006] S1, obtaining energy usage data of the industrial park in real time through the source / grid / load / storage model, wherein the energy usage data includes status information of power source, grid, load, and energy storage equipment;

[0007] S2, preprocessing the energy usage data;

[0008] S3, determines whether the external power supply of the industrial park fails. If so, the pre-processed energy usage data is substituted into the deep reinforcement learning model to generate an action data set of the energy storage power supply.

[0009] In some embodiments of the present invention, the source / grid / load / storage model includes an internal model and an external model.

[0010] In some embodiments of the present invention, the internal model is as follows:

[0011] S self ={SoC,T}

[0012] (E min ≤E≤E max )

[0013] Where: S self is the internal model of the system, SoC is the charge state of the reserve power supply, E max is the battery capacity in a fully charged state, E max is the battery capacity at the lowest state of charge; T is the temperature of the energy storage power source itself.

[0014] In some embodiments of the present invention, the external model is as follows:

[0015] S other = {P L ,P AC ,P W ,P PV}

[0016] Where: S other is the external model of the system, P L is the load demand, P AC Active power provided by AC, P W Active power provided by wind turbine, P PV Active power provided to photovoltaics.

[0017] In certain embodiments of the present invention, the deep reinforcement learning model includes: a joint distribution rate of a visible layer and a hidden layer, a marginal distribution function, and an activation probability.

[0018] In some embodiments of the present invention, the joint distribution rate of the visible layer and the hidden layer is calculated as follows:

[0019]

[0020] Where: θ={W ij ,a i ,b j} are the parameters of the restricted Boltzmann machine, W ij represents the weight between the visible unit and the hidden layer, a i ,b j Represent the bias of the visible layer and the hidden layer respectively;

[0021]

[0022]

[0023] Where: P(v,h|θ) is the joint distribution rate of the visible layer and the hidden layer, and Z(θ) is the normalization function of the data.

[0024] In some embodiments of the present invention, the marginal distribution function is calculated as follows:

[0025]

[0026] Where: P(v|θ) is the marginal distribution function.

[0027] In some embodiments of the present invention, the activation probability is calculated as follows:

[0028]

[0029] Where: P(h j =1|v,θ) is the activation probability of the jth hidden unit in the hidden layer, is the activation function si gmoi d;

[0030]

[0031] Where: P(v j =1|h,θ) is the activation probability of the jth visible unit in the visible layer.

[0032] In certain embodiments of the present invention, preprocessing the energy usage data includes: data cleaning, and removing outliers and noise.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The present invention uses deep reinforcement learning to control the energy storage backup power supply of the industrial park, so that when the external power supply of the microgrid fails, the energy storage backup power supply can provide stable energy for the microgrid. The proposed method can effectively manage and control the energy storage backup power supply of the industrial park, and has good economic benefits and practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flow chart of a method for controlling an energy storage backup power supply in an industrial park according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] See also Figure 1 This embodiment provides an industrial park energy storage backup power supply control method based on improved deep reinforcement learning, comprising the following steps:

[0038] S1, through the source / grid / load / storage model, the energy usage data of the industrial park is obtained in real time. The energy usage data includes the status information of the power source, grid, load, and energy storage equipment. Specifically, it is as follows: the operating parameters of the source (such as solar panels, wind turbines and other renewable energy power generation facilities), grid (grid power supply), load (energy consumption of each load in the park), and storage (the status of the energy storage system, such as the power level of the battery energy storage system). These data provide a comprehensive overview of the operating environment for the control method and are the basis for subsequent decision-making.

[0039] Specifically, the source / grid / load / storage model includes an internal model and an external model.

[0040] The internal model is as follows:

[0041] S self ={SoC,T}

[0042] (E min ≤E≤E max )

[0043] Where: S self is the internal model of the system, SoC is the charge state of the reserve power supply, E max is the battery capacity in a fully charged state, E max is the battery capacity at the lowest state of charge; T is the temperature of the energy storage power source itself.

[0044] The external model is as follows:

[0045] S other = {P L ,P AC ,P W ,P PV}

[0046] Where: S other is the external model of the system, P L is the load demand, P AC Active power provided by AC, P W Active power provided by wind turbine, P PV Active power provided to photovoltaics.

[0047] S2, preprocessing the energy usage data. Specifically, the data is cleaned to remove outliers and noise to ensure the accuracy and reliability of the data. Normalization processing can also be performed to make data of different dimensions comparable, which is convenient for subsequent analysis and processing.

[0048] S3, determines whether the external power supply of the industrial park fails. If so, the pre-processed energy usage data is substituted into the deep reinforcement learning model to generate an action data set of the energy storage power supply.

[0049] The deep reinforcement learning model includes: the joint distribution rate of the visible layer and the hidden layer, the marginal distribution function and the activation probability, which are described in detail below.

[0050] The joint distribution rate of the visible layer and the hidden layer is calculated as follows:

[0051]

[0052] Where: θ={W ij ,a i ,b j} are the parameters of the restricted Boltzmann machine, W ij represents the weight between the visible unit and the hidden layer, a i ,b j Represent the bias of the visible layer and the hidden layer respectively;

[0053]

[0054]

[0055] Where: P(v,h|θ) is the joint distribution rate of the visible layer and the hidden layer, and Z(θ) is the normalization function of the data.

[0056] The marginal distribution function is calculated as follows:

[0057]

[0058] Where: P(v|θ) is the marginal distribution function.

[0059] The activation probability is calculated as follows:

[0060]

[0061] Where: P(h j =1|v,θ) is the activation probability of the jth hidden unit in the hidden layer, is the activation function si gmoi d;

[0062]

[0063] Where: P(v j =1|h,θ) is the activation probability of the jth visible unit in the visible layer.

[0064] Taking the data of an industrial park in a city in Zhejiang Province on a certain day as an example, the intelligent control method for energy storage backup power supply in the industrial park based on the improved deep reinforcement learning algorithm described in the present invention is deployed in the power monitoring center of the industrial park. The data changes before and after the deployment are compared. As shown in Table 1:

[0065] Table 1

[0066]

[0067] The power data of each key node in the park is obtained in real time. Due to extreme weather, the external power grid fails. This method quickly identifies this state change and triggers step S3. The deep reinforcement learning model is immediately started, and based on historical data and the current system status, the action data of the energy storage power supply is quickly generated, instructing the energy storage power supply to start discharging to maintain the power supply of key loads in the park. After receiving the action data, the execution module immediately controls the energy storage power supply to perform the discharge operation, successfully ensuring the power supply stability of the park during the external power failure.

[0068] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, the embodiments should be considered exemplary and non-restrictive in all respects, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present invention.

Claims

1. A method for controlling energy storage backup power supply in an industrial park based on improved deep reinforcement learning, characterized in that: The following steps are involved: S1, obtaining energy usage data of the industrial park in real time through the source / grid / load / storage model, wherein the energy usage data includes status information of power source, grid, load, and energy storage equipment; S2, preprocessing the energy usage data; S3, determines whether the external power supply of the industrial park fails. If so, the pre-processed energy usage data is substituted into the deep reinforcement learning model to generate an action data set of the energy storage power supply.

2. The industrial park energy storage backup power supply control method based on improved deep reinforcement learning according to claim 1 is characterized in that: The source / grid / load / storage model includes an internal model and an external model.

3. The industrial park energy storage backup power supply control method based on improved deep reinforcement learning according to claim 2 is characterized in that: The internal model is as follows: S self ={SoC,T} Where: S self is the internal model of the system, SoC is the charge state of the reserve power supply, E max is the battery capacity in a fully charged state, E max is the battery capacity at the lowest state of charge; T is the temperature of the energy storage power source itself.

4. The industrial park energy storage backup power supply control method based on improved deep reinforcement learning according to claim 2 is characterized in that: The external model is as follows: S other ={P L ,P AC ,P W ,P PV } Where: S other is the external model of the system, P L is the load demand, P AC Active power provided by AC, P W Active power provided to wind turbines, P PV Active power provided to photovoltaics.

5. The industrial park energy storage backup power supply control method based on improved deep reinforcement learning according to claim 1 is characterized in that: The deep reinforcement learning model includes: a joint distribution rate of a visible layer and a hidden layer, a marginal distribution function and an activation probability.

6. The industrial park energy storage backup power supply control method based on improved deep reinforcement learning according to claim 5 is characterized in that: The joint distribution rate of the visible layer and the hidden layer is calculated as follows: Where: θ={W ij ,a i ,b j } are the parameters of the restricted Boltzmann machine, W ij represents the weight between the visible unit and the hidden layer, a i ,b j Represent the bias of the visible layer and the hidden layer respectively; Where: P(v,h|θ) is the joint distribution rate of the visible layer and the hidden layer, and Z(θ) is the normalization function of the data.

7. The industrial park energy storage backup power supply control method based on improved deep reinforcement learning according to claim 6 is characterized in that: The marginal distribution function is calculated as follows: Where: P(v|θ) is the marginal distribution function.

8. The industrial park energy storage backup power supply control method based on improved deep reinforcement learning according to claim 7 is characterized in that: The activation probability is calculated as follows: Where: P(h j =1|v,θ) is the activation probability of the jth hidden unit in the hidden layer, is the activation function sigmoid; Where: P(v j =1|h,θ) is the activation probability of the jth visible unit in the visible layer.

9. The industrial park energy storage backup power supply control method based on improved deep reinforcement learning according to claim 1 is characterized in that: Preprocessing the energy usage data includes: data cleaning, and removing outliers and noise.

Images