Method and device for adjusting integrated energy system and electronic equipment

Abstract

The invention discloses an adjusting method and device for an integrated energy system and electronic equipment, and relates to the technical field of electrics and new energies, and the method comprises the steps: firstly, determining a first factor and a second factor according to the system sampling information of the integrated energy system, and then determining a reward function according to the constraint information of the integrated energy system, then, the integrated energy system is adjusted according to a preset algorithm, the first factor, the second factor and the reward function, and the preset algorithm is used for determining the range of parameters for adjusting the integrated energy system. According to the method and the device, the technical problem that the accuracy of adjusting the comprehensive energy system is low due to the fact that the coupling degree among multiple energy sources in the comprehensive energy system in the prior art is increased is solved.

Description

Technical Field

[0001] The present application relates to the fields of electrical technology and new energy technology, and more specifically, to a regulation method, device and electronic equipment for an integrated energy system. Background Art

[0002] In the existing electrical technology field, interconnected power grids, distributed power sources, new energy systems and power-consuming buildings are usually managed as an integrated energy system. With the development of new energy technologies, photovoltaic power generation, wind power generation or cogeneration power generation equipment is installed on power-consuming buildings to achieve the purpose of supplying energy to the integrated energy system. Power-consuming buildings are not only energy consumers in the integrated energy system, but also energy producers in the integrated energy system.

[0003] In order to improve the utilization efficiency of new energy in the integrated energy system, it is necessary to dispatch different types of energy in the integrated energy system. As the access ratio of distributed new energy in the integrated energy system continues to increase, the coupling degree between various energy sources in the integrated energy system is also increasing, resulting in the problem of increased fluctuations in renewable energy and grid load in the new energy system. In order to improve the stability of the integrated energy system during operation, static strategies are usually used in the prior art to adjust the integrated energy system, that is, to optimize and dispatch the system in a fixed time period. However, static strategies cannot cope with real-time changes in the state of the integrated energy system, resulting in a technical problem of low accuracy in adjusting the integrated energy system.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The present application provides a method, device and electronic device for regulating an integrated energy system, so as to at least solve the technical problem of low accuracy in regulating the integrated energy system caused by the increased coupling between multiple energy sources in the integrated energy system in the prior art.

[0006] According to one aspect of the present application, a method for regulating an integrated energy system is provided, comprising: determining a first factor and a second factor based on system sampling information of the integrated energy system, wherein the integrated energy system includes at least a power grid, a distributed power source, an energy storage system and electrical equipment, the first factor is used to characterize the system state of the integrated energy system, and the second factor is used to characterize the decision information on which the system state of the integrated energy system is regulated; determining a reward function based on constraint information of the integrated energy system, wherein the constraint information includes at least an electric power balance condition, a thermal power balance condition and a performance limitation condition of the energy storage system of the integrated energy system, and the reward function is used to measure the performance of the integrated energy system; regulating the integrated energy system based on a preset algorithm, the first factor, the second factor and the reward function, wherein the preset algorithm is used to determine the range of parameters for regulating the integrated energy system.

[0007] Optionally, in the process of determining the first factor based on the system sampling information of the integrated energy system, the adjustment method of the integrated energy system also includes: determining the actual power purchase power and load information of the power grid at the current moment based on the system sampling information; determining the first power, second power, and third power of the integrated energy system at the current moment based on the system sampling information, wherein the first power is used to characterize the output power of the distributed power source, the second power is used to characterize the charging power and discharging power of the battery system, and the third power is used to characterize the heat absorption power and heat release power of the heat storage system; determining the first factor based on the actual power purchase power, load information, the first power, the second power, and the third power.

[0008] Optionally, in the process of determining the second factor based on the system sampling information of the integrated energy system, the adjustment method of the integrated energy system also includes: determining a fourth power based on the system sampling information and a preset strategy, wherein the fourth power is used to characterize the output power of the distributed power source at a future moment; determining a planned power purchase power of the power grid at a future moment based on the system sampling information and the preset strategy; and determining the second factor based on the planned power purchase power and the fourth power.

[0009] Optionally, in the process of determining the reward function based on the constraint information of the integrated energy system, the adjustment method of the integrated energy system also includes: determining the target cost of the integrated energy system based on the power purchase cost of the power grid, the operating cost of the distributed power source, the load compensation cost of the power grid, and the fuel cost of the gas units in the energy storage system; and determining the reward function based on the target cost and the constraint information.

[0010] Optionally, before adjusting the integrated energy system according to the preset algorithm, the first factor, the second factor and the reward function, the adjustment method of the integrated energy system also includes: storing the first factor, the second factor and the reward function collected within a preset time period in a database connected to the target network and the preset network, the target network is a commentator network generated based on the preset algorithm, the preset network is used to determine the strategy distribution information according to the preset strategy, and the strategy distribution information is used to characterize the probability of using the decision information represented by the second factor to adjust the integrated energy system.

[0011] Optionally, after storing the first factor, the second factor and the reward function collected within a preset time period in a database connected to the target network and the preset network, the adjustment method of the integrated energy system also includes: inputting the first factor stored in the database into the target network to obtain a state value function corresponding to the first factor, wherein the state value function is used to characterize the state type of the system state corresponding to the first factor; determining the first loss function based on the state value function and the discount factor corresponding to the first factor, wherein the first loss function is used to perform reverse transmission updates on the target network.

[0012] Optionally, after storing the first factor, the second factor and the reward function collected within the preset time period in a database connected to the target network and the preset network, the adjustment method of the integrated energy system also includes: inputting the second factor stored in the database into the preset network to obtain the strategy distribution information corresponding to the second factor; determining the second loss function based on the strategy distribution information corresponding to the second factor and the second factor, wherein the second loss function is used to perform reverse transmission update on the preset network.

[0013] According to another aspect of the present application, a regulating device for an integrated energy system is also provided, including: a first determination unit, used to determine a first factor and a second factor based on system sampling information of the integrated energy system, wherein the integrated energy system includes at least a power grid, a distributed power source, an energy storage system and electrical equipment, the first factor is used to characterize the system state of the integrated energy system, and the second factor is used to characterize the decision information relied on for regulating the system state of the integrated energy system; a second determination unit, used to determine a reward function based on constraint information of the integrated energy system, wherein the constraint information includes at least an electric power balance condition, a thermal power balance condition and a performance limitation condition of the energy storage system of the integrated energy system, and the reward function is used to measure the performance of the integrated energy system; an regulating unit, used to regulate the integrated energy system based on a preset algorithm, the first factor, the second factor and the reward function, wherein the preset algorithm is used to determine the range of parameters for regulating the integrated energy system.

[0014] According to another aspect of the present application, a computer program product is also provided, in which a computer program is stored, wherein when the computer program is running, the computer program product is controlled to execute any one of the above-mentioned methods for regulating the integrated energy system.

[0015] According to another aspect of the present application, an electronic device is also provided, wherein the electronic device includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement any one of the above-mentioned methods for regulating the integrated energy system.

[0016] In the present application, firstly, a first factor and a second factor are determined based on the system sampling information of the integrated energy system, wherein the integrated energy system includes at least a power grid, a distributed power source, an energy storage system and electrical equipment, and the first factor is used to characterize the system state of the integrated energy system, and the second factor is used to characterize the decision information on which the system state of the integrated energy system is adjusted. Then, the present application determines a reward function based on the constraint information of the integrated energy system, wherein the constraint information includes at least the electric power balance condition, the thermal power balance condition and the performance limitation condition of the energy storage system of the integrated energy system, and the reward function is used to measure the performance of the integrated energy system. Finally, the present application adjusts the integrated energy system based on a preset algorithm, the first factor, the second factor and the reward function, wherein the preset algorithm is used to determine the range of parameters for adjusting the integrated energy system.

[0017] From the above content, it can be seen that the technical solution of the present application takes the system state of the integrated energy system as the first factor, and the decision information on which the system state of the integrated energy system is adjusted as the second factor. In addition, the present application also determines a reward function for measuring the performance of the integrated energy system based on the constraint information of the integrated energy system. The present application can determine the real-time status information of the integrated energy system through the reward function, the first factor and the second factor, thereby solving the problem in the prior art that the static strategy used to adjust the integrated energy system cannot cope with the real-time changes in the state of the integrated energy system.

[0018] In addition, the present application adjusts the integrated energy system based on a preset algorithm, a first factor, a second factor and a reward function, and the preset algorithm in the present application can determine the range of parameters for adjusting the integrated energy system, thereby avoiding the problem of too large an adjustment range for the energy system, causing instability in the energy system. In addition, the reward function in the present application is determined based on the electric power balance condition, thermal power balance condition and performance limitation condition of the energy storage system of the integrated energy system. Therefore, the technical solution of the present application can ensure that the electric power and thermal power of the system are in a balanced state during the adjustment of the system, thereby further improving the accuracy of the system adjustment, thereby solving the problem of low accuracy in adjusting the integrated energy system in the prior art.

[0019] It can be seen that in this application, the purpose of adjusting the integrated energy system based on the real-time status information of the integrated energy system is achieved by adjusting the integrated energy system based on the preset algorithm, the first factor, the second factor and the reward function, thereby achieving the technical effect of improving the accuracy of system adjustment, and further solving the technical problem of low accuracy in adjusting the integrated energy system caused by the increased coupling between multiple energy sources in the integrated energy system in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0021] Figure 1 is a flow chart of an optional method for regulating a comprehensive energy system according to an embodiment of the present application;

[0022] Figure 2 is a schematic diagram of an optional integrated energy system according to an embodiment of the present application;

[0023] Figure 3 is a flow chart of an optional method for adjusting an integrated energy system based on a PPO algorithm according to an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of an optional regulating device of an integrated energy system according to an embodiment of the present application;

[0025] Figure 5 is a schematic diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] It should also be noted that the relevant information (including the target user's login information and relevant information of system resources) and data (including but not limited to data for display and analysis) involved in this application are all information and data authorized by the user or fully authorized by all parties. For example, an interface is set up between this system and the relevant user or organization. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0029] In addition, the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant information and data involved in this application comply with the relevant laws, regulations and standards of the relevant regions, and necessary confidentiality measures have been taken and do not violate public order and good morals. In addition, this application provides corresponding operation entrances for users to choose to agree to authorization or refuse authorization. If the user chooses to refuse authorization, he / she will enter the corresponding expert decision-making process.

[0030] According to an embodiment of the present application, an embodiment of a method for regulating an integrated energy system is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0031] The present application provides a regulation system for an integrated energy system (hereinafter referred to as the regulation system) for executing the regulation method for the integrated energy system in the present application. Figure 1 is a flow chart of an optional integrated energy system regulation method according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0032] Step S101 : determining a first factor and a second factor according to system sampling information of an integrated energy system.

[0033] In step S101, the integrated energy system includes at least a power grid, a distributed power source, an energy storage system and electrical equipment. The first factor is used to characterize the system state of the integrated energy system, and the second factor is used to characterize the decision information relied on for adjusting the system state of the integrated energy system.

[0034] Optionally, the distributed power source includes wind power generation equipment, photovoltaic power generation equipment, traditional gas turbine power generation equipment and power generation equipment composed of a heat recovery system.

[0035] Optionally, the energy storage system includes batteries and heat storage tanks. The present application solves the problems of wind power abandonment and solar power abandonment caused by the randomness and volatility of new energy power generation equipment by introducing the energy storage system into the comprehensive energy system, thereby realizing the on-site consumption of renewable new energy. In addition, the regulation system can reduce the operating cost of the system by improving the energy efficiency ratio of energy conversion.

[0036] Step S102, determining a reward function according to constraint information of the integrated energy system.

[0037] In step S102, the constraint information includes at least the electric power balance condition, the thermal power balance condition and the performance limitation condition of the energy storage system of the integrated energy system, and the reward function is used to measure the performance of the integrated energy system.

[0038] Optionally, the performance constraint conditions of the energy storage system include at least battery constraint conditions and heat storage tank constraint conditions.

[0039] Step S103, adjusting the integrated energy system according to the preset algorithm, the first factor, the second factor and the reward function.

[0040] In step S103, a preset algorithm is used to determine a range of parameters for adjusting the integrated energy system.

[0041] Optionally, the present application can train the regulation system based on a preset algorithm, wherein the preset algorithm is preferably a proximal strategy optimization algorithm. The proximal strategy optimization algorithm limits the amplitude of the update strategy in the regulation system, thereby ensuring the stability and convergence of the regulation system during training, thereby improving the accuracy of subsequent regulation of the integrated energy system in a complex state. In addition, the proximal strategy optimization algorithm is used to solve the optimal scheduling problem of the integrated energy system, which can achieve the purpose of regulating the integrated energy system under different environmental conditions, thereby improving the flexibility of the regulation system during application.

[0042] From the above content, it can be seen that the technical solution of the present application takes the system state of the integrated energy system as the first factor, and the decision information on which the system state of the integrated energy system is adjusted as the second factor. In addition, the present application also determines a reward function for measuring the performance of the integrated energy system based on the constraint information of the integrated energy system. The present application can determine the real-time status information of the integrated energy system through the reward function, the first factor and the second factor, thereby solving the problem in the prior art that the static strategy used to adjust the integrated energy system cannot cope with the real-time changes in the state of the integrated energy system.

[0043] In addition, the present application adjusts the integrated energy system based on a preset algorithm, a first factor, a second factor and a reward function, and the preset algorithm in the present application can determine the range of parameters for adjusting the integrated energy system, thereby avoiding the problem of too large an adjustment range for the energy system, causing instability in the energy system. In addition, the reward function in the present application is determined based on the electric power balance condition, thermal power balance condition and performance limitation condition of the energy storage system of the integrated energy system. Therefore, the technical solution of the present application can ensure that the electric power and thermal power of the system are in a balanced state during the adjustment of the system, thereby further improving the accuracy of the system adjustment, thereby solving the problem of low accuracy in adjusting the integrated energy system in the prior art.

[0044] It can be seen that in this application, the purpose of adjusting the integrated energy system based on the real-time status information of the integrated energy system is achieved by adjusting the integrated energy system based on the preset algorithm, the first factor, the second factor and the reward function, thereby achieving the technical effect of improving the accuracy of system adjustment, and further solving the technical problem of low accuracy in adjusting the integrated energy system caused by the increased coupling between multiple energy sources in the integrated energy system in the prior art.

[0045] Optionally, Figure 2 is a schematic diagram of an optional integrated energy system according to an embodiment of the present application, such as Figure 2As shown in the figure, the energy input of the integrated energy system includes grid electricity, distributed power output and natural gas, among which the distributed power includes wind power generation equipment, photovoltaic power generation equipment, traditional gas turbine power generation equipment and power generation equipment composed of a heat recovery system. The power output is completed by the transformer and the gas turbine power generation equipment, and the heat output is completed by the heat recovery system in the gas turbine and the gas boiler. The energy storage system in the integrated energy system includes batteries and heat storage tanks. The load of the integrated energy system includes uncontrollable rigid loads and controllable flexible loads.

[0046] In an optional embodiment, the regulation system first determines the actual power purchase power and load information of the power grid at the current moment based on the system sampling information. Then, the regulation system determines the first power, second power, and third power of the integrated energy system at the current moment based on the system sampling information, wherein the first power is used to characterize the output power of the distributed power source, the second power is used to characterize the charging power and discharging power of the battery system, and the third power is used to characterize the heat absorption power and heat release power of the heat storage system. Then, the regulation system determines the first factor based on the actual power purchase power, load information, the first power, the second power, and the third power.

[0047] Optionally, the calculation process of the regulating system to determine the first factor is shown in the following formula (1):

[0048] s t ={t,P net (t),P MT (t),P w (t),P pv (t),P GB (t),P L (t),P HST (t),P BAT (t)} (1)

[0049] In the above formula (1), S t is the first factor, t represents the current moment, P net (t) is the actual power purchase power of the power grid at the current moment (i.e., the exchange power of the power grid), P MT (t) is the rated output power of the gas turbine at the current moment, P w (t) is the output power of the wind power generation equipment at the current moment, P pv (t) is the output power of the photovoltaic power generation equipment at the current moment, P GB (t) is the actual output power of the gas turbine at the current moment, P L (t) represents the flexible load of the integrated energy system at the current moment, P HST (t) is the heat absorption and release power of the heat storage tank at the current moment, P BAT (t) is the charging and discharging power of the battery at the current moment.

[0050] In an optional embodiment, the regulation system first determines a fourth power based on system sampling information and a preset strategy, wherein the fourth power is used to characterize the output power of the distributed power source at a future moment. Then, the regulation system determines a planned power purchase power of the power grid at a future moment based on the system sampling information and the preset strategy. Finally, the regulation system determines a second factor based on the planned power purchase power and the fourth power.

[0051] Optionally, the calculation process of the regulating system to determine the second factor is shown in the following formula (2):

[0052] a t = {P net (t+1),P MT (t+1),P w (t+1),P pv (t+1),P HST (t+1),P BAT (t+1)} (2)

[0053] In the above formula (2), a t is the second factor, t+1 represents the next future moment, P net (t+1) is the planned power purchase power of the power grid at the future time, P MT (t+1) is the rated output power of the gas turbine at the future time, P w (t+1) is the output power of the wind power generation equipment at the future time, P pv (t+1) is the output power of the photovoltaic power generation equipment at the future time, P HST (t+1) is the heat absorption and release power of the heat storage tank at the future time, P BAT (t+1) is the charging and discharging power of the battery at the future time.

[0054] In an optional embodiment, the regulation system first determines the target cost of the integrated energy system based on the power purchase cost of the power grid, the operating cost of the distributed power sources, the load compensation cost of the power grid, and the fuel cost of the gas units in the energy storage system, and then the regulation system determines the reward function based on the target cost and constraint information.

[0055] Optionally, the calculation process of the target cost of the integrated energy system is shown in the following formula (3):

[0056] f cost =F wpg +F net +F HB +F L +F MT (3)

[0057] In the above formula (3), minF cost represents the minimum target cost of the integrated energy system, F wpg represents the operating costs of wind power generation equipment, photovoltaic power generation equipment and gas turbines, F net is the power purchase cost of the power grid, F HB Represents the cost of heat storage tank and battery consumption, F L Represents the total compensation cost for optimization of electrical and thermal flexible loads, F MT In order to reduce the operating cost of the gas turbine, the regulation system adjusts the integrated energy system based on a preset algorithm to achieve the purpose of reducing the target cost of the integrated energy system.

[0058] Optionally, F wpg 、F net 、F HB 、F MT and F L The calculation process is shown in the following formula (4):

[0059]

[0060] In the above formula (4), K w , K pv , K GB Represent the operating cost coefficients of wind power generation equipment, photovoltaic power generation equipment and gas turbine respectively, K b represents the time-of-use electricity price when purchasing electricity from the power grid, K HST and K BAT Represent the operating cost coefficients of the heat storage tank and battery, K MT Represents the rated operating factor of the gas turbine, F shift 、F tran and F cut They represent the costs incurred by the shiftable load, transferable load and curtailable load in the integrated energy system respectively.

[0061] Optionally, the constraint information of the integrated energy system includes at least the electric power balance condition, the thermal power balance condition and the performance limitation condition of the energy storage system, wherein the performance limitation condition of the energy storage system includes at least the battery constraint condition and the heat storage tank constraint condition.

[0062] Specifically, the process of determining the electric power balance condition of the integrated energy system is shown in the following formula (5):

[0063]

[0064] In the above formula (5), P output Represents the sum of the electric power output of wind turbines, photovoltaics, gas turbines and the power grid, L eis the total electrical load, L base is the rigid load in the building, L shift , L tran and L cut They represent the load that can be translated, the load that can be transferred, and the load that can be reduced in the integrated energy system, P w,max is the maximum output of the wind turbine, P pv,max is the maximum output of the photovoltaic group, P net,min is the minimum value of the grid exchange power, P net,max is the maximum value of the grid exchange power, P MT,max is the rated electrical power of the gas turbine, P BAT,min is the minimum power of battery charging and discharging, P BAT,max The maximum power for charging and discharging the battery.

[0065] Specifically, the determination process of the thermal power balance condition of the integrated energy system is shown in the following formula (6):

[0066]

[0067] In the above formula (6), Q L is the total heat load, Q HT and Q GB are the heat loads required to meet the indoor temperature and indoor water temperature, respectively, HT,max is the rated output thermal power of the heat recovery system, Q GB,max is the rated thermal output power of the gas turbine, Q HST,min is the minimum power of heat storage tank, Q HST,max The maximum power for dissipating heat from the heat storage tank.

[0068] Specifically, the battery constraints of the integrated energy system are as follows:

[0069] As shown in the following formula (7), the battery state of charge S soc Satisfy the upper and lower limit constraints:

[0070] S min ≤S soc (t)≤S max (7)

[0071] In the above formula (7), S min S is the lower limit of the battery charge state during period t, max is the upper limit of the battery's state of charge during period t.

[0072] As shown in the following formula (8), within a unit time interval, the battery cannot be in the charging and discharging state at the same time, so the charging and discharging state of the battery must meet the following constraints:

[0073] Xt ·Y t =0 (8)

[0074] In the above formula (8), X t is the charging state, X t ∈{0,1},Y t is the discharge state, Y t ∈{0,1}.

[0075] As shown in the following formula (9), the number of times the battery is charged and discharged in a scheduling cycle will affect the battery life, so the following constraints are added:

[0076]

[0077] In the above formula (9), N1 and N2 are the number of battery charge and discharge limits, T represents the total scheduling time, and ε is the activation function.

[0078] As shown in the following formula (10), when the battery participates in the system operation optimization process, its energy state must satisfy the constraint of being equal at the beginning and end of the scheduling cycle:

[0079] S0=S T (10)

[0080] In the above formula (10), S0 and S T They are the initial state of charge and final state of charge of the battery respectively.

[0081] Specifically, the constraints of the heat storage tank of the integrated energy system are as follows:

[0082] As shown in the following formula (11), the heat storage tank cannot be in the heat absorption and heat release state at the same time, so the heat absorption state A of the heat storage tank T and exothermic state B T The following constraints must be met:

[0083] A t ·B t =0 (11)

[0084] As shown in the following formula (12), the energy state of the heat storage tank must also satisfy the constraint of being equal at the beginning and end of the scheduling cycle:

[0085] W0=W T (12)

[0086] In the above formula (12), W0 and W T They are the initial heat storage tank heat and the final heat storage tank heat of the battery respectively.

[0087] In an optional embodiment, the regulation system stores the first factor, the second factor and the reward function collected within a preset time period in a database connected to the target network and the preset network, the target network is a commentator network generated based on a preset algorithm, and the preset network is used to determine the strategy distribution information according to the preset strategy, and the strategy distribution information is used to characterize the probability of using the decision information represented by the second factor to adjust the integrated energy system.

[0088] Optionally, Figure 3 is a flowchart of an optional method for adjusting an integrated energy system based on a PPO (Proximal Policy Optimization) algorithm according to an embodiment of the present application, such as Figure 3 As shown, the PPO algorithm is essentially a Markov decision process, and its decision process can be used to include five elements.<S,a,P,r,γ> It is represented by a five-tuple, where S is the state space (corresponding to the first factor mentioned above), a is the action space (corresponding to the second factor mentioned above), P is the state transition probability, r is the reward function, and γ is the discount factor.

[0089] Specifically, the calculation formula of the state space at the current moment (i.e., moment t) is shown in the above formula (1), and the calculation formula of the action space at the next future moment (i.e., moment t+1) is shown in the above formula (2).

[0090] Moreover, the reward function represents the instant reward obtained by the integrated energy system according to the current state space and decision-making actions, and is the standard for measuring the quality of the intelligent agent's behavior. The scheduling goal of the integrated energy system is to reduce the total operating cost of the integrated energy system (i.e., target cost) as much as possible while satisfying the constraints (i.e., constraint information) during the operation of each device. Therefore, the calculation formula of the target cost is used as the reward item, and the constraint information is used as the penalty item. The reward function obtained based on the reward item and the penalty item is shown in the following formula (13):

[0091]

[0092] In the above formula (13), e f With e c are the penalty coefficient and target cost coefficient respectively, F all,t is the penalty function, β is the positive transformation coefficient, when F all,t >0, the reward function is negative, and the system reduces the state transition probability of the action space when updating the strategy; when F all,t =0, the reward function is positive, and the system is encouraged to increase the state transition probability of the action space when updating the strategy, ultimately achieving the goal of maximizing the reward.

[0093] In an optional embodiment, after storing the first factor, the second factor and the reward function collected within a preset time period in a database connected to the target network and the preset network, the adjustment system first inputs the first factor stored in the database into the target network to obtain a state value function corresponding to the first factor, wherein the state value function is used to characterize the state type of the system state corresponding to the first factor. Thereafter, the adjustment system determines a first loss function based on the state value function corresponding to the first factor and a discount factor, wherein the first loss function is used to perform reverse transmission updates on the target network.

[0094] In an optional embodiment, after storing the first factor, the second factor and the reward function collected within a preset time period in a database connected to the target network and the preset network, the adjustment system first inputs the second factor stored in the database into the preset network to obtain the strategy distribution information corresponding to the second factor, and then the adjustment system determines the second loss function based on the strategy distribution information corresponding to the second factor and the second factor, wherein the second loss function is used to perform reverse transmission updates on the preset network.

[0095] Alternatively, if Figure 3 As shown, the PPO algorithm uses a deep neural network to represent the strategy function, wherein the strategy function is used to guide the integrated energy system to take corresponding actions according to environmental information. The PPO algorithm limits the range of change of the strategy parameters. When updating the strategy, it can avoid the problem of excessive update amplitude causing performance degradation of the integrated energy system environment, thereby improving the stability of the strategy update process and improving the performance of the integrated energy system environment.

[0096] Furthermore, the target network (i.e. Figure 3 The update steps of the Critic network in are as follows:

[0097] Step 1: The regulation system interacts with the integrated energy system. The regulation system converts the state space corresponding to each decision step (i.e. Figure 3 S0 to S T-1 ), action space (i.e. Figure 3 a0 to a T-1 ) and the reward function (i.e. Figure 3 r0 to r T-1 ) are stored in the memory until the current scheduling cycle is completed (i.e. Figure 3 In T), at this time, the target network is not updated.

[0098] Step 2: The state space corresponding to the last decision step stored in the memory bank (corresponding to Figure 3 S T ) is input into the target network to obtain the state value function V(S) of the state space corresponding to the last decision step. T); The discounted reward Q(S t ,a t ); Input the state spaces corresponding to all decision steps stored in the memory bank into the target network, and obtain the state value function corresponding to each state space (i.e. Figure 3 V(S0) to V(S T-1 )); The advantage function corresponding to each state space is calculated based on the discounted reward and the state value function corresponding to each state space (i.e. Figure 3 A(S0) to A(S T-1 )).

[0099] Step 3: Determine the first loss function corresponding to the target network based on the advantage function corresponding to the state space, and then back propagate to update the Critic Network.

[0100] Optionally, the first loss function is shown in the following formula (14):

[0101] c_loss=E[Q(s t ,a t )-V(s t )] 2 (14)

[0102] In the above formula (14), c_loss represents the first loss function, A t =Q(S t ,a t )-V(S t ), which represents the advantage function corresponding to the state space.

[0103] Step 4: Input the state space corresponding to all decision steps stored in the memory bank into the preset network to obtain the new strategy distribution (i.e. Figure 3 N θ (μ,σ 2 ), μ is the mean of the distribution, σ 2 is the distribution variance) and the old strategy distribution (i.e. Figure 3 N θold (μ,σ 2 )), where the new policy distribution is the action probability distribution generated by the policy network that is currently being trained and updated, and the old policy distribution is the action probability distribution generated by the previous version of the policy network.

[0104] Specifically, the preset network includes a new action network (i.e. Figure 3 "Actor-New") and the old action network (i.e. Figure 3The new action network is used to determine the actions taken by the integrated energy system in the current state space, and the new action network iteratively updates the strategy to maximize the cumulative reward; the old action network is the version before the new action network strategy is updated. The old action network is used to determine the target value and the importance sampling ratio, and the old action network provides a benchmark for the update of the new action network, thereby avoiding the problem of excessive update gradient of the new action network.

[0105] After obtaining the new strategy distribution and the old strategy distribution, the adjustment system inputs the decision space corresponding to all decision steps stored in the memory bank into the new strategy distribution and the old strategy distribution respectively, and calculates the probability value of each action being selected in the new and old strategies, that is, the new strategy probability (corresponding to Figure 3 π θ (a t |S t )) and the old strategy probability (corresponding to Figure 3 π θold (a t |S t )), then, the new strategy probability is divided by the old strategy probability to get the probability ratio r t (θ).

[0106] Step 5, determine the value of the objective function according to the ratio of the advantage function and the probability corresponding to the state space (the objective function is shown in the above formula (1)), and determine the second loss function corresponding to the preset network according to the negative value of the objective function.

[0107] Optionally, the second loss function is shown in the following formula (15):

[0108] a_loss=E[min(r t (θ)A t , F cost ((r t (θ),1-ε,1+ε)A t )] (15)

[0110] In the above formula (15), a_loss represents the second loss function,

[0111] Compared with the prior art, the advantages of the above embodiment are as follows:

[0112] The regulation system of the integrated energy system based on the proximal strategy optimization algorithm limits the amplitude of the strategy update, thereby ensuring the stability and convergence of the neural network (target network and preset network) during training, making the neural network more reliable during training, and helping to deal with complex integrated energy system scheduling problems; the proximal strategy optimization algorithm is used to solve the optimal scheduling problem of the integrated energy system, thereby achieving the purpose of optimizing the scheduling of the integrated energy system under different environmental conditions. In other words, even in the face of unknown system states and environmental conditions, the neural network trained based on the proximal strategy optimization algorithm can also show good performance, thereby providing greater flexibility for the practical application of the regulation system of the integrated energy system.

[0113] From the above content, it can be seen that the technical solution of the present application takes the system state of the integrated energy system as the first factor, and the decision information on which the system state of the integrated energy system is adjusted as the second factor. In addition, the present application also determines a reward function for measuring the performance of the integrated energy system based on the constraint information of the integrated energy system. The present application can determine the real-time status information of the integrated energy system through the reward function, the first factor and the second factor, thereby solving the problem in the prior art that the static strategy used to adjust the integrated energy system cannot cope with the real-time changes in the state of the integrated energy system.

[0114] In addition, the present application adjusts the integrated energy system based on a preset algorithm, a first factor, a second factor and a reward function, and the preset algorithm in the present application can determine the range of parameters for adjusting the integrated energy system, thereby avoiding the problem of too large an adjustment range for the energy system, causing instability in the energy system. In addition, the reward function in the present application is determined based on the electric power balance condition, thermal power balance condition and performance limitation condition of the energy storage system of the integrated energy system. Therefore, the technical solution of the present application can ensure that the electric power and thermal power of the system are in a balanced state during the adjustment of the system, thereby further improving the accuracy of the system adjustment, thereby solving the problem of low accuracy in adjusting the integrated energy system in the prior art.

[0115] It can be seen that in this application, the purpose of adjusting the integrated energy system based on the real-time status information of the integrated energy system is achieved by adjusting the integrated energy system based on the preset algorithm, the first factor, the second factor and the reward function, thereby achieving the technical effect of improving the accuracy of system adjustment, and further solving the technical problem of low accuracy in adjusting the integrated energy system caused by the increased coupling between multiple energy sources in the integrated energy system in the prior art.

[0116] According to another aspect of the embodiment of the present application, a regulating device for an integrated energy system is also provided. Figure 4is a schematic diagram of an optional regulating device of an integrated energy system according to an embodiment of the present application, such as Figure 4 As shown, the regulating device of the integrated energy system includes: a first determining unit 401 , a second determining unit 402 and a regulating unit 403 .

[0117] Optionally, a first determination unit is used to determine a first factor and a second factor based on system sampling information of an integrated energy system, wherein the integrated energy system includes at least a power grid, a distributed power source, an energy storage system and electrical equipment, the first factor is used to characterize the system state of the integrated energy system, and the second factor is used to characterize the decision information on which the system state of the integrated energy system is adjusted; a second determination unit is used to determine a reward function based on constraint information of the integrated energy system, wherein the constraint information includes at least an electric power balance condition, a thermal power balance condition and a performance limitation condition of the energy storage system of the integrated energy system, and the reward function is used to measure the performance of the integrated energy system; an adjustment unit is used to adjust the integrated energy system based on a preset algorithm, the first factor, the second factor and the reward function, wherein the preset algorithm is used to determine the range of parameters for adjusting the integrated energy system.

[0118] In an optional embodiment, the first determining unit includes: a first determining subunit, a second determining subunit, and a third determining subunit.

[0119] Optionally, the first determination subunit is used to determine the actual purchased power and load information of the power grid at the current moment based on the system sampling information; the second determination subunit is used to determine the first power, second power, and third power of the comprehensive energy system at the current moment based on the system sampling information, wherein the first power is used to characterize the output power of the distributed power source, the second power is used to characterize the charging power and discharging power of the battery system, and the third power is used to characterize the heat absorption power and heat release power of the heat storage system; the third determination subunit is used to determine the first factor based on the actual purchased power, load information, the first power, the second power, and the third power.

[0120] In an optional embodiment, the first determining unit further includes: a fourth determining subunit, a fifth determining subunit and a sixth determining subunit.

[0121] Optionally, the fourth determination subunit is used to determine a fourth power based on the system sampling information and a preset strategy, wherein the fourth power is used to characterize the output power of the distributed power source at a future moment; the fifth determination subunit is used to determine the planned power purchase power of the power grid at a future moment based on the system sampling information and a preset strategy; the sixth determination subunit is used to determine the second factor based on the planned power purchase power and the fourth power.

[0122] In an optional embodiment, the second determining unit includes: a seventh determining subunit and an eighth determining subunit.

[0123] Optionally, the seventh determination subunit is used to determine the target cost of the integrated energy system based on the power purchase cost of the power grid, the operating cost of the distributed power source, the load compensation cost of the power grid, and the fuel cost of the gas units in the energy storage system; the eighth determination subunit is used to determine the reward function based on the target cost and constraint information.

[0124] In an optional embodiment, the regulating device of the integrated energy system further includes: a storage unit.

[0125] Optionally, the storage unit is used to store the first factor, the second factor and the reward function collected within a preset time period in a database connected to the target network and the preset network, the target network is a critic network generated based on a preset algorithm, the preset network is used to determine the strategy distribution information according to the preset strategy, and the strategy distribution information is used to characterize the probability of using the decision information represented by the second factor to adjust the integrated energy system.

[0126] In an optional embodiment, the regulating device of the integrated energy system further includes: a first input unit and a third determination unit.

[0127] Optionally, a first input unit is used to input a first factor stored in a database into a target network to obtain a state value function corresponding to the first factor, wherein the state value function is used to characterize a state type of a system state corresponding to the first factor; and a third determination unit is used to determine a first loss function based on the state value function corresponding to the first factor and a discount factor, wherein the first loss function is used to perform reverse transfer updates on the target network.

[0128] In an optional embodiment, the regulating device of the integrated energy system further includes: a second input unit and a fourth determination unit.

[0129] Optionally, a second input unit is used to input the second factor stored in the database into the preset network to obtain strategy distribution information corresponding to the second factor; a fourth determination unit is used to determine a second loss function based on the strategy distribution information corresponding to the second factor and the second factor, wherein the second loss function is used to perform reverse transmission update on the preset network.

[0130] From the above content, it can be seen that the technical solution of the present application takes the system state of the integrated energy system as the first factor, and the decision information on which the system state of the integrated energy system is adjusted as the second factor. In addition, the present application also determines a reward function for measuring the performance of the integrated energy system based on the constraint information of the integrated energy system. The present application can determine the real-time status information of the integrated energy system through the reward function, the first factor and the second factor, thereby solving the problem in the prior art that the static strategy used to adjust the integrated energy system cannot cope with the real-time changes in the state of the integrated energy system.

[0131] In addition, the present application adjusts the integrated energy system based on a preset algorithm, a first factor, a second factor and a reward function, and the preset algorithm in the present application can determine the range of parameters for adjusting the integrated energy system, thereby avoiding the problem of too large an adjustment range for the energy system, causing instability in the energy system. In addition, the reward function in the present application is determined based on the electric power balance condition, thermal power balance condition and performance limitation condition of the energy storage system of the integrated energy system. Therefore, the technical solution of the present application can ensure that the electric power and thermal power of the system are in a balanced state during the adjustment of the system, thereby further improving the accuracy of the system adjustment, thereby solving the problem of low accuracy in adjusting the integrated energy system in the prior art.

[0132] It can be seen that in this application, the purpose of adjusting the integrated energy system based on the real-time status information of the integrated energy system is achieved by adjusting the integrated energy system based on the preset algorithm, the first factor, the second factor and the reward function, thereby achieving the technical effect of improving the accuracy of system adjustment, and further solving the technical problem of low accuracy in adjusting the integrated energy system caused by the increased coupling between multiple energy sources in the integrated energy system in the prior art.

[0133] According to another aspect of an embodiment of the present application, a computer program product is further provided, the computer program product comprising a stored computer program, wherein when the computer program is running, the computer program product is controlled to execute any one of the above-mentioned methods for regulating an integrated energy system.

[0134] According to another aspect of an embodiment of the present application, there is also provided an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned methods for regulating an integrated energy system by executing the executable instructions.

[0135] Optionally, Figure 5 is a schematic diagram of an optional electronic device according to an embodiment of the present application, such as Figure 5As shown, an embodiment of the present application provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, any one of the above-mentioned methods for regulating the integrated energy system is implemented.

[0136] The above-mentioned embodiments or examples disclosed in the present application are not exhaustive, but are only illustrative of some embodiments or examples, and are not intended to be specific limitations on the scope of protection disclosed in the present application. In the absence of contradiction, each step in a certain embodiment or example in the present application can be implemented as an independent example, and the steps can be combined arbitrarily. For example, the scheme after removing some steps in a certain embodiment or example can also be implemented as an independent example, and the order of the steps in a certain embodiment or example can be arbitrarily exchanged. In addition, the optional methods or optional examples in a certain embodiment or example can be combined arbitrarily; in addition, the various embodiments or examples can be combined arbitrarily, for example, some or all steps of different embodiments or examples can be combined arbitrarily, and a certain embodiment or example can be combined arbitrarily with the optional methods or optional examples of other embodiments or examples.

[0137] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0138] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0139] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0140] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0141] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory. The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0142] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0143] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0144] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0145] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for regulating an integrated energy system, characterized in that: include: Determine a first factor and a second factor based on system sampling information of an integrated energy system, wherein the integrated energy system includes at least a power grid, a distributed power source, an energy storage system, and electrical equipment, the first factor is used to characterize a system state of the integrated energy system, and the second factor is used to characterize decision information relied upon for adjusting the system state of the integrated energy system; Determine a reward function according to constraint information of the integrated energy system, wherein the constraint information includes at least an electric power balance condition, a thermal power balance condition and a performance limit condition of an energy storage system of the integrated energy system, and the reward function is used to measure the performance of the integrated energy system; The integrated energy system is adjusted according to a preset algorithm, the first factor, the second factor and the reward function, wherein the preset algorithm is used to determine a range of parameters for adjusting the integrated energy system.

2. The method for regulating the integrated energy system according to claim 1, characterized in that: The first factor is determined based on the system sampling information of the integrated energy system, including: Determine the actual purchased power and load information of the power grid at the current moment according to the system sampling information; Determine the first power, the second power, and the third power of the integrated energy system at the current moment according to the system sampling information, wherein the first power is used to characterize the output power of the distributed power source, the second power is used to characterize the charging power and the discharging power of the battery system, and the third power is used to characterize the heat absorption power and the heat release power of the heat storage system; The first factor is determined according to the actual purchased power, the load information, the first power, the second power, and the third power.

3. The method for regulating the integrated energy system according to claim 1, characterized in that: The second factor is determined based on the system sampling information of the integrated energy system, including: Determine a fourth power according to the system sampling information and a preset strategy, wherein the fourth power is used to characterize the output power of the distributed power source at a future moment; Determine the planned power purchase power of the power grid at the future moment according to the system sampling information and the preset strategy; The second factor is determined according to the planned purchased power and the fourth power.

4. The method for regulating the integrated energy system according to claim 1, characterized in that: Determining a reward function according to the constraint information of the integrated energy system includes: Determine the target cost of the integrated energy system according to the power purchase cost of the power grid, the operating cost of the distributed power source, the load compensation cost of the power grid and the fuel cost of the gas unit in the energy storage system; The reward function is determined according to the target cost and the constraint information.

5. The method for regulating the integrated energy system according to claim 1, characterized in that: Before adjusting the integrated energy system according to the preset algorithm, the first factor, the second factor and the reward function, the adjustment method of the integrated energy system further includes: The first factor, the second factor and the reward function collected within a preset time period are stored in a database connected to a target network and a preset network, wherein the target network is a critic network generated based on the preset algorithm, and the preset network is used to determine strategy distribution information according to a preset strategy, and the strategy distribution information is used to characterize the probability of adjusting the integrated energy system using the decision information represented by the second factor.

6. The method for regulating the integrated energy system according to claim 5, characterized in that: After storing the first factor, the second factor and the reward function collected within the preset time period in a database connected to the target network and the preset network, the adjustment method of the integrated energy system further includes: Inputting the first factor stored in the database into the target network to obtain a state value function corresponding to the first factor, wherein the state value function is used to characterize the state type of the system state corresponding to the first factor; A first loss function is determined according to the state value function and the discount factor corresponding to the first factor, wherein the first loss function is used to perform a reverse propagation update on the target network.

7. The method for regulating the integrated energy system according to claim 5, characterized in that: After storing the first factor, the second factor and the reward function collected within the preset time period in a database connected to the target network and the preset network, the adjustment method of the integrated energy system further includes: Inputting the second factor stored in the database into the preset network to obtain policy distribution information corresponding to the second factor; A second loss function is determined according to the strategy distribution information corresponding to the second factor and the second factor, wherein the second loss function is used to perform a reverse propagation update on the preset network.

8. A regulating device for an integrated energy system, characterized in that: include: A first determination unit is used to determine a first factor and a second factor according to system sampling information of an integrated energy system, wherein the integrated energy system includes at least a power grid, a distributed power source, an energy storage system, and electrical equipment, the first factor is used to characterize a system state of the integrated energy system, and the second factor is used to characterize decision information on which the system state of the integrated energy system is adjusted; a second determination unit, configured to determine a reward function according to constraint information of the integrated energy system, wherein the constraint information at least includes an electric power balance condition, a thermal power balance condition, and a performance limit condition of an energy storage system of the integrated energy system, and the reward function is used to measure the performance of the integrated energy system; An adjustment unit is used to adjust the integrated energy system according to a preset algorithm, the first factor, the second factor and the reward function, wherein the preset algorithm is used to determine the range of parameters for adjusting the integrated energy system.

9. A computer program product, characterized in that The computer program product comprises a computer program, wherein when the computer program is run, the computer program product is controlled to execute the method for regulating the integrated energy system according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the regulation method of the integrated energy system as described in any one of claims 1 to 7.

Images