An energy management method for an integrated energy system under demand response

Abstract

The present invention belongs to the field of energy management technology and proposes an energy management method for an integrated energy system under demand response, comprising the following steps: Step 1: Establishing an energy equipment model, a user energy consumption behavior model, and an operation optimization model for the integrated energy system; Step 2: Constructing the neural network required for TD3; Step 3: Based on the model established in Step 1, using the energy management center to interact with the integrated energy system and putting the historical information obtained from the interaction into the experience pool, while using TD3 to optimize the strategy network and value network; Step 4: Based on the optimized strategy network obtained in Step 3, controlling the integrated energy system so that it can participate in power demand response. The present invention helps the integrated energy system to rationally arrange the production plans of internal energy equipment and the energy consumption plans of internal users, further improving the system's demand response potential, and maintaining the safe, stable, and economical operation of the integrated energy system while completing the power grid demand response plan.

Description

Technical Field

[0001] The present invention belongs to the field of energy management technology, and more specifically, relates to an energy management method for an integrated energy system under demand response. Background Art

[0002] Environmental pollution and energy shortages are becoming increasingly serious worldwide, creating an urgent need to improve energy efficiency. Integrated energy systems, based on coupled multi-energy production and coordinated dispatching of multiple energy networks, break the existing model of separate planning and independent operation of each energy source and achieve coordinated optimization and mutual assistance among multiple energy sources. Power demand response is a key method for implementing power demand management. It primarily guides users to adjust their electricity usage through electricity prices and policy incentives, effectively alleviating power supply constraints and improving the security and stability of grid operations. With the gradual implementation of power demand policies in provinces such as Guangdong and Anhui, integrated energy systems are now able to participate in power demand response. They can adjust their energy production plans and other methods to alter energy demand on the grid, demonstrating significant potential for demand response regulation.

[0003] Solutions to the optimization and scheduling of integrated energy systems primarily involve nonlinear methods, intelligent algorithms, and linear simplification. The first two methods suffer from lengthy computation times for solving high-dimensional nonlinear problems with increasingly tight coupling between electricity, heat, and cooling, making them difficult to meet online computational requirements. While linearization is easier to compute and solve, it inevitably leads to large errors in the resulting solutions. With the rise and development of artificial intelligence (AI), deep reinforcement learning (DL) has gained increasing attention in the optimization and control of integrated energy systems. DL possesses powerful autonomous learning capabilities and can draw on historical experience from vast amounts of historical data. Intelligent agents can take different actions under different system states and learn from rewards to achieve optimal strategies. The entire process of interacting with the environment does not rely on detailed and precise model information, making it easier to implement in real-world scheduling scenarios than traditional methods. DL also utilizes offline learning and online decision-making to enable real-time system operational decisions. Summary of the Invention

[0004] In response to the problems existing in the existing technology of energy management of integrated energy systems, the present invention proposes an energy management method for integrated energy systems under demand response, which can achieve safe, stable and economical operation of the integrated energy system under the influence of multiple random factors when participating in power demand response.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] An energy management method for an integrated energy system under demand response includes an integrated energy system, wherein the integrated energy system includes an energy management center (EMS), energy conversion equipment, energy storage equipment, and multiple types of users. The energy conversion equipment includes a gas turbine (GT), a waste heat boiler (HB), a gas boiler (GB), an absorption chiller (AR), an electric heat pump (EB), an electric refrigeration unit (ER), and a photovoltaic power generation device (PV). The energy storage equipment includes a battery (ES) and a heat storage tank (HS). The multiple types of users include public service users and commercial users. The user loads include three energy forms: electrical load, thermal load, and cooling load. The energy management center is an energy and information integration center that externally connects with electricity suppliers and natural gas suppliers, can obtain information such as electricity prices, natural gas prices, and demand response plans issued by power companies, and control the purchase amount of electricity and natural gas energy. Internally, it can issue response invitations to multiple types of users to adjust their inherent energy consumption behavior and control the production plans of energy equipment within the system, thereby participating in and completing the power company's power demand response, while taking into account the safety and economy of system operation. The energy management method is characterized by specifically comprising the following steps:

[0007] Step 1: The integrated energy system participates in power demand response with one day as a dispatch cycle. The overall goal of its optimized operation is to maximize the total profit of the system operation in one day on the basis of completing the power demand response target. Therefore, it is necessary to establish an operation optimization model for the integrated energy system participating in power demand response:

[0008] Step 1.1: Establish a model for the power demand response plan, PV output randomness, outdoor temperature randomness, and user heating and cooling load randomness.

[0009] Step 1.2: Establish a scheduling model for energy conversion equipment and energy storage equipment in an integrated energy system;

[0010] Step 1.3: Establish a multi-class user response characteristic model within the integrated energy system;

[0011] Step 1.4: Establish a dispatch optimization model for the integrated energy system participating in power demand response;

[0012] Step 2: Build the deep neural network required for the Twin Delayed Deep Deterministic Policy Gradient Algorithm (TD3):

[0013] TD3 has three independent neural networks: the policy network, value network 1, and value network 2. Each network has its own target network: the target policy network, target value network 1, and target value network 2. The above six neural networks are all fully connected neural networks, including input layers, hidden layers, and output layers. The policy network and target policy network have the same structure, and the value network 1, value network 2, target value network 1, and target value network 2 have the same structure.

[0014] Step 3: Based on the model established in step 1, the energy management center interacts with the integrated energy system to obtain historical interaction information and store it in the experience pool. TD3 is used to iteratively update and optimize the policy network, value network 1, and value network 2.

[0015] Step 4: Implement control of the integrated energy system based on the strategy network obtained in step 3:

[0016] The feasibility of the verification algorithm is verified using the integrated energy system established above. At any decision-making moment in the system operation, the decision-making moment, invitation response amount, and operating status of the integrated energy system are normalized and input into the strategy network of TD3. After forward propagation, the action information output by the strategy network is denormalized to obtain the actions that the system can take at the current decision moment. After the integrated energy system repeatedly performs the above operations within a scheduling cycle, the completion status of the system's power demand response and the energy sales revenue, response revenue, energy purchase cost, incentive cost, maintenance cost, and carbon tax cost obtained are observed.

[0017] Furthermore, the energy management method for an integrated energy system under demand response is characterized in that the power demand response plan, photovoltaic output randomness, outdoor temperature randomness, and user electric heating and cooling load randomness model described in step 1.1 are specifically implemented as follows:

[0018] Determine the amount of demand response required to participate in electricity demand response:

[0019] Determine the decision time t of the integrated energy system in a scheduling cycle k The agreed response quantity P′ of the power demand response peak (t k ), and the demand response target of the integrated energy system at each decision moment in a scheduling cycle is obtained:

[0020] {P′ peak (t1),P′ peak (t1),...,P′ peak (t k ),...,P′ peak (t K )}

[0021] Photovoltaic output randomness:

[0022] Photovoltaic output error ΔP pv Subject to the mean μ pv The variance is σ pv Normal distribution, that is, ΔP pv ~N(μ pv ,σ pv 2 ), its probability density function can be expressed as:

[0023]

[0024] If the decision time t k The predicted value of photovoltaic output is P pv,0 (t k ), after adding random errors, the actual value of photovoltaic output P can be obtained pv (t k )for:

[0025] P pv (t k )=P pv,0 (t k )+ΔP pv

[0026] Temperature randomness:

[0027] Temperature error ΔT temp Subject to the mean μ temp The variance is σ temp Normal distribution, that is, ΔT temp ~N(μ temp ,σ temp 2 ), its probability density function can be expressed as:

[0028]

[0029] If the decision time t k The predicted temperature is T temp,0 (t k ), after adding random errors, the actual temperature value T temp (t k )for:

[0030] T temp (t k )=T temp,0 (t k )+ΔT temp

[0031] Electric load randomness:

[0032] Error of electrical load ΔL eleSubject to the mean μ ele The variance is σ ele Normal distribution, that is, ΔL ele ~N(μ ele ,σ ele 2 ), its probability density function can be expressed as:

[0033]

[0034] If the decision time t k The predicted value of the electric load is L ele,0 (t k ), after adding random errors, the actual value of the load can be obtained L ele (t k )for:

[0035] L ele (t k )=L ele,0 (t k )+ΔL ele

[0036] Heat load randomness:

[0037] Thermal load error ΔL hot Subject to the mean μ hot The variance is σ hot Normal distribution, that is, ΔL hot ~N(μ hot ,σ hot 2 ), its probability density function can be expressed as:

[0038]

[0039] If the decision time t k The predicted value of heat load is L hot,0 (t k ), after adding random errors, the actual value of the heat load can be obtained L hot (t k )for:

[0040] L hot (t k )=L hot,0 (t k )+ΔL hot

[0041] Cooling load randomness:

[0042] Error of cooling load ΔL cold Subject to the mean μ cold The variance is σ cold Normal distribution, that is, ΔL cold~N(μ cold ,σ cold 2 ), its probability density function can be expressed as:

[0043]

[0044] If the decision time t k The predicted value of cooling load is L cold,0 (t k ), after adding random errors, the actual value of cooling load L cold (t k )for:

[0045] L cold (t k )=L cold,0 (t k )+ΔL cold

[0046] Furthermore, the energy management method for an integrated energy system under demand response is characterized in that the scheduling model of energy conversion equipment and energy storage equipment of the integrated energy system described in step 1.2 comprises the following specific steps:

[0047] Gas turbine dispatch process:

[0048] The gas turbine consumes natural gas to provide electricity and heat. The decision time t k The electric power output of the gas turbine is calculated as follows:

[0049] P gt (t k )=V gt (t k )H ng η gt

[0050] Among them, P gt (t k ) and V gt (t k ) is the decision time t k The electrical power output and natural gas consumed by the gas turbine; H ng is the calorific value of natural gas; η gt is the power generation efficiency of the gas turbine.

[0051] The ratio of the gas turbine output thermal power to electrical power is the thermal-to-electric ratio b, which can be expressed as:

[0052]

[0053] Among them, Q gt (t k ) is the decision time tk The heating power of the gas turbine.

[0054] Waste heat boiler scheduling process:

[0055] Waste heat boilers can improve energy utilization efficiency and provide thermal energy to users by absorbing waste heat from high-temperature flue gas discharged by gas turbines. Its mathematical model is:

[0056] Q hb (t k )=Q hb,0 (t k )η hb

[0057] Among them, Q hb (t k ) and Q hb,0 (t k )Decision time t k Thermal power absorbed and output by waste heat boiler; η hb is the waste heat recovery efficiency.

[0058] Gas boiler scheduling process:

[0059] Gas boiler consumes natural gas to provide heat energy. At the decision time t k The output thermal power is calculated as follows:

[0060] Q gb (t k )=V gb (t k )H ng η gb

[0061] Among them, Q gb (t k ) and V gb (t k ) is the decision time t k The thermal power output of the gas boiler and the natural gas consumed; η gb is the heating efficiency of the gas boiler.

[0062] Absorption chiller scheduling process:

[0063] The absorption chiller provides cooling power by absorbing heat power. k The output cooling power is calculated as follows:

[0064] H ar (t k )=Q ar (t k )η ar

[0065] Among them, H ar (t k ) and Q ar (t k ) is the decision time t k The cooling power output and heating power absorbed by the absorption chiller; η ar is the cooling efficiency of the absorption chiller.

[0066] Electric heat pump scheduling process:

[0067] The electric heat pump converts low-quality heat energy into high-quality heat energy by consuming electricity. k The thermal power output of the electric heat pump is calculated as follows:

[0068] Q eb (t k )=P eb (t k )η eb

[0069] Among them, Q eb (t k ) and P eb (t k ) is the decision time t k The thermal power output and electrical power consumed by the electric heat pump; η eb is the heating efficiency of the electric heat pump.

[0070] Electric refrigeration unit scheduling process:

[0071] The electric refrigeration unit provides cooling power by consuming electric power. The decision time t k The cooling power output of the electric refrigeration unit is calculated as follows:

[0072] H er (t k )=P er (t k )η er

[0073] Among them, H er (t k ) and P er (t k ) is the decision time t k The cooling power output and electrical power consumed by the electric refrigeration unit; η er is the efficiency of the electric refrigeration unit.

[0074] Photovoltaic power generation device scheduling process:

[0075] Photovoltaic power generation devices use solar energy to generate electricity, and their power generation can be expressed as:

[0076]

[0077] Among them, P pv (t k ) and G light (t k ) is the decision time t k The power generation of the photovoltaic power generation device and the light intensity at this time; p stc and G stc is the power generation capacity and corresponding light intensity of the photovoltaic power generation device under standard conditions.

[0078] Battery scheduling process:

[0079] Define the decision time t k The state of charge of the battery is SOC es (t k ), which represents the percentage of the battery's remaining charge to its rated capacity. Taking into account the battery's power dissipation and charging and discharging during use, the battery's dynamic charging and discharging process is described as follows:

[0080] SOC es (t k )=(1-σ es )SOC es (t k -1)Δt-η es P es (t k )Δt / V es

[0081] Where Δt is the time difference between two decision moments, σ es is the battery energy loss rate; P es (t k ) is the decision time t k Battery charge and discharge power, positive value represents discharge, negative value represents charge, 0 represents idle state; V es is the capacity of the battery; η es is the battery charge and discharge coefficient, which can be expressed as:

[0082]

[0083] in, and It is the charging and discharging efficiency of the battery.

[0084] Thermal storage tank scheduling process:

[0085] Based on the above description of the battery state of charge, the decision time t is defined as k The thermal energy storage state of the heat storage tank is SOC hs (t k ), the dynamic storage and discharge process of the heat storage tank is described as follows:

[0086] SOC hs (t k )=(1-σ hs )SOC hs (t k -1)Δt-η hs Q hs (t k )Δt / V hs

[0087] Among them, σ hs is the energy loss rate of the heat storage tank; Q hs (t k ) is the decision time t k The heat storage tank heat storage and release power, positive value represents heat release, negative value represents heat storage, 0 represents idle state; V hs is the capacity of the heat storage tank; η hs is the heat storage and release coefficient of the heat storage tank, which can be expressed as:

[0088]

[0089] in, and is the heat storage and release efficiency of the heat storage tank.

[0090] Furthermore, the energy management method for an integrated energy system under demand response is characterized in that the multi-class user response characteristic model within the integrated energy system described in step 1.3 comprises the following specific steps:

[0091] Public service user response characteristic model:

[0092] Public service users refer to users such as office buildings and hospital buildings that serve residents' production and life. They have a large amount of non-essential loads, which can be reduced when necessary. The load reduction model for public service users is as follows:

[0093]

[0094]

[0095] in, and P cut (t k ) is the decision time t k The original power of the load that can be reduced and the power after reduction, α cut (t k ) is the user decision time t k Reduction ratio of load that can be reduced, ε cut- and ε cut+The critical compensation price a and b are the load reduction coefficients when the user is willing to reduce load and reach the upper limit of reduction capacity.

[0096] The integrated energy system operator issues a subsidy price to the user, and the user reduces the load that can be reduced according to the subsidy price. The user makes a decision at the decision time t k Compensation amount C obtained after load reduction cut (t k ) can be expressed as:

[0097]

[0098] Business User Response Characteristic Model:

[0099] Commercial users include large supermarkets and shopping malls, which have a large amount of air conditioning load. The modeling of air conditioning load includes the thermodynamic modeling of the building to which the air conditioning is located and the electric heat / cooling conversion of the air conditioning. The thermodynamic model of the building to which the air conditioning is located usually adopts an equivalent thermal parameter model based on circuit simulation. The discrete form of the differential equation expression of the first-order equivalent thermal parameter model is:

[0100]

[0101] Among them, T a (t k ) and T o (t k ) is the decision time t k Indoor and outdoor air temperature, R a and C a is the equivalent thermal resistance and equivalent heat capacity of indoor air, P ac (t k ) and H ac (t k ) is the decision time t k Air conditioning power and cooling power, η ac For air conditioning cooling efficiency.

[0102] Assume that the user sets the temperature to the most comfortable temperature T when not participating in the response a,fit , we can get the decision time t through the above formula k Air conditioning power consumption P when the user does not participate in the response ac,fit (t k )for:

[0103]

[0104] The indoor temperature can be maintained within the acceptable range for human body[T a,min ,T a,max], the air conditioner is allowed to participate in the dispatching, and the integrated energy system operator signs a contract with the user to agree on the adjustment of the compensation price ε for the air conditioner power consumption. ac , adjust the power consumption of the air conditioner when necessary, and the user makes the decision at the time t k The amount of compensation received is:

[0105] C ac (t k )=ε ac |P ac (t k )-P ac,fit (t k )|Δt

[0106] Furthermore, the energy management method for an integrated energy system under demand response is characterized in that the integrated energy system described in step 1.4 participates in the scheduling optimization model for power demand response, and the specific steps are:

[0107] After receiving the peak load regulation order from the upper power grid, the integrated energy system operator formulates the operation plan of each device and issues dispatch instructions to users to achieve the response target and maximize its own profit R, while ensuring the safe and stable operation of the system. The objective function is expressed as follows:

[0108]

[0109] Among them, I sell (t k ), I resp (t k ), C buy (t k ), C insp (t k ), C mc (t k ), C Ctax (t k ) are respectively at the decision time t k The energy sales revenue, response revenue, energy purchase cost, incentive cost, maintenance cost and carbon tax cost of the integrated energy system operator; K is the total number of decision moments in the entire scheduling cycle.

[0110] Energy sales revenue is the sum of the revenue from electricity sales, heat sales, and cooling sales to users by the integrated energy system operator, which can be expressed as:

[0111] I sell (t k )=[p ele (t k )L ele (t k )+p hot (tk )L hot (t k )+p cold (t k )L cold (t k )]Δt

[0112] Among them, L ele (t k ), L hot (t k ) and L cold (t k ) are the decision time t k The user's electrical load, heating load and cooling load power; p ele (t k ), p hot (t k ) and p cold (t k ) are the decision time t k The prices of electricity, heat and cooling sold to users.

[0113] The response benefit is the economic compensation obtained by the integrated energy system after responding to the power demand of the grid, which can be expressed as:

[0114] I resp (t k )=γ(t k )ε peak (t k )|P peak (t k )|Δt

[0115] Among them, γ(t k ) and ε peak (t k ) is the decision time t k The load response rate coefficient and compensation price of power demand response, γ(t k ) and decision time t k The load response rate is related to P peak (t k ) is the decision time t k The actual response power of the integrated energy system. A positive value represents valley filling, and a negative value represents peak shaving.

[0116] The energy purchase cost is the sum of the costs of electricity and natural gas purchased by the integrated energy system operator from the power grid and natural gas grid, which can be expressed as:

[0117] C buy (t k )=p ele (t k )P grid(t k )Δt+p gas (t k )V gas (t k )

[0118] Among them, P grid (t k ) and V gas (t k ) is the system at the decision time t k Electric power and natural gas volume purchased, p ele (t k ) and p gas (t k ) is the decision time t k External electricity and natural gas prices.

[0119] The incentive cost is the total compensation cost given by the integrated energy system operator to users who participate in the integrated demand response, which can be expressed as:

[0120] C insp (t k )=C cut (t k )+C ac (t k )

[0121] Maintenance cost refers to the maintenance cost of various internal equipment during operation by the integrated energy system operator, which can be expressed as:

[0122]

[0123] Among them, c mc,n and P n (t k ) is the unit power maintenance cost of equipment n and the decision time t k Output of device n, where N is the total number of devices.

[0124] The carbon tax cost is the environmental protection fee levied by the environmental protection department due to the pollution caused by the system during operation, which can be expressed as:

[0125]

[0126] Among them, ω Ctax is the carbon tax coefficient, E gas and E grid is the CO2 emission per unit of natural gas and electricity, η grid The transmission efficiency of the power grid.

[0127] Furthermore, the energy management method for an integrated energy system under demand response is characterized in that the iterative update and optimization of the strategy network, value network 1 and value network 2 by using TD3 in step 3 are specifically performed as follows:

[0128] Step 3.1: Initialize learning and decision parameters, including: initializing the number of decision moments K in a day; initializing the number of learning steps STEP; initializing the number of learning steps step = 0; initializing the sample pool capacity and learning sample batch size M and Batch; initializing the discount factor γ and the soft update coefficient τ; initializing the network parameter update cycle Cycle; initializing the neural network, including: initializing the parameters θ of the policy network, value network 1 and value network 2 respectively π 、 and Initialize the parameters θ of the target policy network, target value network 1, and target value network 2 π′ ←θ π 、 and

[0129] Step 3.2: Initialize the decision time k=0, initialize the integrated energy system state, process the system operation data through the energy management center, and store the generated samples in the experience pool:

[0130] Step 3.2.1, the state s k Normalize to get Will Input the current policy network to get the corresponding action π(s k |θ π ), superimpose noise v1 to obtain random action a k , that is: a k =π(s k |θ π )+v1, execute the currently selected action a k , after a decision cycle, the system reaches the next state s k+1 , and observe the running cost r in the process k , get the sample [s k ,a k ,r k ,s k+1 ], and stored in the experience pool after normalization;

[0131] Step 3.2.2: Set k = k + 1. If k < K, return to step 3.2.1. If k = K, proceed to step 3.3.

[0132] Step 3.3: Randomly select a batch of quadruple sample data from the experience pool

[0133] Step 3.4: Obtain the state through the target policy network The target action To improve the robustness of the training process, motion noise is superimposed on the target action. Get random target action Right now:

[0134]

[0135] Step 3.5: Calculate the state through the value network Next action The corresponding value function and

[0136] Step 3.6, through the target value network, get the state Random target action The corresponding target value function and According to the Bellman equation, the state Next action The corresponding target value function Q target ,Right now:

[0137] Step 3.7, update the network parameters by minimizing the loss function. The loss function It can be expressed as:

[0138] Step 3.8: Set step = step + 1. If step = N l *Cycle,N l ∈Z + , then the policy network is updated through deterministic policy gradient, which can be expressed as:

[0139]

[0140] Parameters θ of target policy network, target value network 1 and target value network 2 π′ 、 and The parameters of the policy network, value network 1, and value network 2 are obtained through soft update and can be expressed as: θ π′ =(1-τ)θ π′ +τθ π 、 If step < STEP, return to step 3.2; if step = STEP, stop learning and the network training is completed.

[0141] Different from the existing technology, the above technical solution has the following beneficial effects:

[0142] 1. While managing the energy of the integrated energy system, the present invention takes into account the randomness of photovoltaic power generation, temperature, and user heating and cooling loads, and uses temperature and subsidy prices as factors affecting user response capabilities and willingness. Based on this, the system can fully tap the user's response potential and the adjustable capabilities of the system's internal energy conversion and energy storage devices.

[0143] 2. The TD3 algorithm selected in this paper is an effective deep reinforcement learning algorithm with strong autonomous learning capabilities and is suitable for high-dimensional continuous action spaces. At the same time, it avoids the Q-value overestimation problem in the actor-critic framework algorithm and improves the training speed and stability of the algorithm.

[0144] 3. The present invention takes the load distribution of multiple types of users within the integrated energy system and the production plans of energy conversion equipment and energy storage equipment as decision variables, and constructs an energy management method for the integrated energy system to participate in power demand response, which is conducive to the safe, stable and economical arrangement of energy distribution in the integrated energy system, further improving the potential of the integrated energy system to participate in power demand response, thereby safely and efficiently completing the demand response plan of the power grid. BRIEF DESCRIPTION OF THE DRAWINGS

[0145] Figure 1 It is the flow chart of energy management and control of integrated energy system;

[0146] Figure 2 It is a block diagram of the composition and energy management of the integrated energy system;

[0147] Figure 3 This is a schematic diagram of energy flow in the integrated energy system;

[0148] Figure 4 Energy management algorithm framework for integrated energy systems based on TD3. DETAILED DESCRIPTION

[0149] In order to explain the technical content, structural features, achieved objectives and effects of the technical solution in detail, the following is a detailed description in conjunction with specific embodiments and accompanying drawings.

[0150] Example 1

[0151] See Figure 2As shown in the figure, the integrated energy system includes an energy management center (EMS), energy conversion equipment, energy storage equipment, and multiple types of users. The energy conversion equipment includes gas turbines (GT), waste heat boilers (HB), gas boilers (GB), absorption chillers (AR), electric heat pumps (EB), electric refrigeration units (ER), and photovoltaic power generation devices (PV). The energy conversion equipment is connected to the power grid via the power bus. Energy storage equipment includes batteries (ES) and heat storage tanks (HS). The multiple types of users mentioned above include public service users and commercial users, and their loads include electricity, heating, and cooling. The energy management center is an energy and information integration center. It connects with electricity and natural gas suppliers externally and can obtain information such as electricity prices, natural gas prices, and demand response plans issued by power companies to control the purchase of electricity and natural gas energy. Internally, it can issue response invitations to multiple types of users to adjust their inherent energy consumption behavior and control the production plans of energy equipment within the system. In this way, it participates in and completes the power company's electricity demand response, while also considering the safety and economic efficiency of system operation.

[0152] See for example Figure 1 As shown, a method for energy management of an integrated energy system under demand response includes the following steps:

[0153] Step 1: The integrated energy system participates in power demand response with one day as a dispatch cycle. The overall goal of its optimized operation is to maximize the total profit of the system operation in one day on the basis of completing the power demand response target. Therefore, it is necessary to establish an operation optimization model for the integrated energy system participating in power demand response:

[0154] Step 1.1: Establish a model for the power demand response plan, PV output randomness, outdoor temperature randomness, and user heating and cooling load randomness.

[0155] Determine the amount of demand response required to participate in electricity demand response:

[0156] Determine the decision time t of the integrated energy system in a scheduling cycle k The agreed response quantity P′ of the power demand response peak (t k ), and the demand response target of the integrated energy system at each decision moment in a scheduling cycle is obtained:

[0157] {P′ peak (t1),P′ peak (t2),...,P′ peak (t k ),...,P′ peak (t K )}

[0158] Photovoltaic output randomness:

[0159] Photovoltaic output error ΔP pv Subject to the mean μ pv The variance is σ pv Normal distribution, that is, ΔP pv ~N(μ pv ,σ pv 2 ), its probability density function can be expressed as:

[0160]

[0161] If the decision time t k The predicted value of photovoltaic output is P pv,0 (t k ), after adding random errors, the actual value of photovoltaic output P can be obtained pv (t k )for:

[0162] P pv (t k )=P pv,0 (t k )+ΔP pv

[0163] Temperature randomness:

[0164] Temperature error ΔT temp Subject to the mean μ temp The variance is σ temp Normal distribution, that is, ΔT temp ~N(μ temp ,σ temp 2 ), its probability density function can be expressed as:

[0165]

[0166] If the decision time t k The predicted temperature is T temp,0 (t k ), after adding random errors, the actual temperature value T temp (t k )for:

[0167] T temp (t k )=T temp,0 (t k )+ΔT temp

[0168] Electric load randomness:

[0169] Error of electrical load ΔL ele Subject to the mean μ ele The variance is σele Normal distribution, that is, ΔL ele ~N(μ ele ,σ ele 2 ), its probability density function can be expressed as:

[0170]

[0171] If the decision time t k The predicted value of the electric load is L ele,0 (t k ), after adding random errors, the actual value of the load can be obtained L ele (t k )for:

[0172] L ele (t k )=L ele,0 (t k )+ΔL ele

[0173] Heat load randomness:

[0174] Thermal load error ΔL hot Subject to the mean μ hot The variance is σ hot Normal distribution, that is, ΔL hot ~N(μ hot ,σ hot 2 ), its probability density function can be expressed as:

[0175]

[0176] If the decision time t k The predicted value of heat load is L hot,0 (t k ), after adding random errors, the actual value of the heat load can be obtained L hot (t k )for:

[0177] L hot (t k )=L hot,0 (t k )+ΔL hot

[0178] Cooling load randomness:

[0179] Error of cooling load ΔL cold Subject to the mean μ cold The variance is σ cold Normal distribution, that is, ΔL cold ~N(μ cold ,σcold 2 ), its probability density function can be expressed as:

[0180]

[0181] If the decision time t k The predicted value of cooling load is L cold,0 (t k ), after adding random errors, the actual value of cooling load L cold (t k )for:

[0182] L cold (t k )=L cold,0 (t k )+ΔL cold

[0183] Step 1.2: Establish a scheduling model for energy conversion equipment and energy storage equipment in an integrated energy system; Figure 3 The figure shows the energy flow process of the integrated energy system. The integrated energy system needs to dispatch its energy conversion equipment and energy storage equipment on the basis of maintaining the supply balance of electrical load, thermal load and cooling load.

[0184] Gas turbine dispatch process:

[0185] The gas turbine consumes natural gas to provide electricity and heat. The decision time t k The electric power output of the gas turbine is calculated as follows:

[0186] P gt (t k )=V gt (t k )H ng η gt

[0187] Among them, P gt (t k ) and V gt (t k ) is the decision time t k The electrical power output and natural gas consumed by the gas turbine; H ng is the calorific value of natural gas; η gt is the power generation efficiency of the gas turbine.

[0188] The ratio of the gas turbine output thermal power to electrical power is the thermal-to-electric ratio b, which can be expressed as:

[0189]

[0190] Among them, Q gt (tk ) is the decision time t k The heating power of the gas turbine.

[0191] Waste heat boiler scheduling process:

[0192] Waste heat boilers can improve energy utilization efficiency and provide thermal energy to users by absorbing waste heat from high-temperature flue gas discharged by gas turbines. Its mathematical model is:

[0193] Q hb (t k )=Q hb,0 (t k )η hb

[0194] Among them, Q hb (t k ) and Q hb,0 (t k )Decision time t k Thermal power absorbed and output by waste heat boiler; η hb is the waste heat recovery efficiency.

[0195] Gas boiler scheduling process:

[0196] Gas boiler consumes natural gas to provide heat energy. At the decision time t k The output thermal power is calculated as follows:

[0197] Q gb (t k )=V gb (t k )H ng η gb

[0198] Among them, Q gb (t k ) and V gb (t k ) is the decision time t k The thermal power output of the gas boiler and the natural gas consumed; η gb is the heating efficiency of the gas boiler.

[0199] Absorption chiller scheduling process:

[0200] The absorption chiller provides cooling power by absorbing heat power. k The output cooling power is calculated as follows:

[0201] H ar (t k )=Q ar (t k )η ar

[0202] Among them, H ar (t k ) and Q ar (t k ) is the decision time t k The cooling power output and heating power absorbed by the absorption chiller; η ar is the cooling efficiency of the absorption chiller.

[0203] Electric heat pump scheduling process:

[0204] The electric heat pump converts low-quality heat energy into high-quality heat energy by consuming electricity. k The thermal power output of the electric heat pump is calculated as follows:

[0205] Q eb (t k )=P eb (t k )η eb

[0206] Among them, Q eb (t k ) and P eb (t k ) is the decision time t k The thermal power output and electrical power consumed by the electric heat pump; η eb is the heating efficiency of the electric heat pump.

[0207] Electric refrigeration unit scheduling process:

[0208] The electric refrigeration unit provides cooling power by consuming electric power. The decision time t k The cooling power output of the electric refrigeration unit is calculated as follows:

[0209] H er (t k )=P er (t k )η er

[0210] Among them, H er (t k ) and P er (t k ) is the decision time t k The cooling power output and electrical power consumed by the electric refrigeration unit; η er is the efficiency of the electric refrigeration unit.

[0211] Photovoltaic power generation device scheduling process:

[0212] Photovoltaic power generation devices use solar energy to generate electricity, and their power generation can be expressed as:

[0213]

[0214] Among them, P pv (t k ) and G light (t k ) is the decision time t k The power generation of the photovoltaic power generation device and the light intensity at this time; p stc and G stc is the power generation capacity and corresponding light intensity of the photovoltaic power generation device under standard conditions.

[0215] Battery scheduling process:

[0216] Define the decision time t k The state of charge of the battery is SOC es (t k ), which represents the percentage of the battery's remaining charge to its rated capacity. Taking into account the battery's power dissipation and charging and discharging during use, the battery's dynamic charging and discharging process is described as follows:

[0217] SOC es (t k )=(1-σ es )SOC es (t k -1)Δt-η es P es (t k )Δt / V es

[0218] Where Δt is the time difference between two decision moments, σ es is the battery energy loss rate; P es (t k ) is the decision time t k Battery charge and discharge power, positive value represents discharge, negative value represents charge, 0 represents idle state; V es is the capacity of the battery; η es is the battery charge and discharge coefficient, which can be expressed as:

[0219]

[0220] in, and It is the charging and discharging efficiency of the battery.

[0221] Thermal storage tank scheduling process:

[0222] Based on the above description of the battery state of charge, the decision time t is defined as k The thermal energy storage state of the heat storage tank is SOC hs (t k), the dynamic storage and discharge process of the heat storage tank is described as follows:

[0223] SOC hs (t k )=(1-σ hs )SOC hs (t k -1)Δt-η hs Q hs (t k )Δt / V hs

[0224] Among them, σ hs is the energy loss rate of the heat storage tank; Q hs (t k ) is the decision time t k The heat storage tank heat storage and release power, positive value represents heat release, negative value represents heat storage, 0 represents idle state; V hs is the capacity of the heat storage tank; η hs is the heat storage and release coefficient of the heat storage tank, which can be expressed as:

[0225]

[0226] in, and is the heat storage and release efficiency of the heat storage tank.

[0227] Step 1.3: Establish a multi-class user response characteristic model within the integrated energy system;

[0228] Public service user response characteristic model:

[0229] Public service users refer to users such as office buildings and hospital buildings that serve residents' production and life. They have a large amount of non-essential loads, which can be reduced when necessary. The load reduction model for public service users is as follows:

[0230]

[0231]

[0232] in, and P cut (t k ) is the decision time t k The original power of the load that can be reduced and the power after reduction, α cut (t k ) is the user decision time t k Reduction ratio of load that can be reduced, ε cut- and ε cut+ is the critical compensation price when the user is willing to reduce load and reaches the upper limit of reduction capacity, a and b are both load reduction coefficients.

[0233] The integrated energy system operator issues a subsidy price to the user, and the user reduces the load that can be reduced according to the subsidy price. The user makes a decision at the decision time t k Compensation amount C obtained after load reduction cut (t k ) can be expressed as:

[0234]

[0235] Business User Response Characteristic Model:

[0236] Commercial users include large supermarkets and shopping malls, which have a large amount of air conditioning load. The modeling of air conditioning load includes the thermodynamic modeling of the building to which the air conditioning is located and the electric heat / cooling conversion of the air conditioning. The thermodynamic model of the building to which the air conditioning is located usually adopts an equivalent thermal parameter model based on circuit simulation. The discrete form of the differential equation expression of the first-order equivalent thermal parameter model is:

[0237]

[0238] Among them, T a (t k ) and T o (t k ) is the decision time t k Indoor and outdoor air temperature, R a and C a is the equivalent thermal resistance and equivalent heat capacity of indoor air, P ac (t k ) and H ac (t k ) is the decision time t k Air conditioning power and cooling power, η ac For air conditioning cooling efficiency.

[0239] Assume that the user sets the temperature to the most comfortable temperature T when not participating in the response a,fit , we can get the decision time t through the above formula k Air conditioning power consumption P when the user does not participate in the response ac,fit (t k )for:

[0240]

[0241] The indoor temperature can be maintained within the acceptable range for human body[T a,min ,T a,max ], the air conditioner is allowed to participate in the dispatching, and the integrated energy system operator signs a contract with the user to agree on the adjustment of the compensation price ε for the air conditioner power consumption. ac, adjust the power consumption of the air conditioner when necessary, and the user makes the decision at the time t k The amount of compensation received is:

[0242] C ac (t k )=ε ac |P ac (t k )-P ac,fit (t k )|Δt

[0243] Step 1.4: Establish a dispatch optimization model for the integrated energy system participating in power demand response;

[0244] After receiving the peak load regulation order from the upper power grid, the integrated energy system operator formulates the operation plan of each device and issues dispatch instructions to users to achieve the response target and maximize its own profit R, while ensuring the safe and stable operation of the system. The objective function is expressed as follows:

[0245]

[0246] Among them, I sell (t k ), I resp (t k ), C buy (t k ), C insp (t k ), C mc (t k ), C Ctax (t k ) are respectively at the decision time t k The energy sales revenue, response revenue, energy purchase cost, incentive cost, maintenance cost and carbon tax cost of the integrated energy system operator; K is the total number of decision moments in the entire scheduling cycle.

[0247] Energy sales revenue is the sum of the revenue from electricity sales, heat sales, and cooling sales to users by the integrated energy system operator, which can be expressed as:

[0248] I sell (t k )=[p ele (t k )L ele (t k )+p hot (t k )L hot (t k )+p cold (t k )L cold (t k )]Δt

[0249] Among them, L ele (t k ), L hot (t k ) and L cold (t k ) are the decision time t k The user's electrical load, heating load and cooling load power; p ele (t k ), p hot (t k ) and p cold (t k ) are the decision time t k The prices of electricity, heat and cooling sold to users.

[0250] The response benefit is the economic compensation obtained by the integrated energy system after responding to the power demand of the grid, which can be expressed as:

[0251] I resp (t k )=γ(t k )ε peak (t k )|P peak (t k )|Δt

[0252] Among them, γ(t k ) and ε peak (t k ) is the decision time t k The load response rate coefficient and compensation price of power demand response, γ(t k ) and decision time t k The load response rate is related to P peak (t k ) is the decision time t k The actual response power of the integrated energy system. A positive value represents valley filling, and a negative value represents peak shaving.

[0253] The energy purchase cost is the sum of the costs of electricity and natural gas purchased by the integrated energy system operator from the power grid and natural gas grid, which can be expressed as:

[0254] C buy (t k )=p ele (t k )P grid (t k )Δt+p gas (t k )V gas (t k )

[0255] Among them, P grid (t k ) and V gas (t k ) is the system at the decision time t k Electric power and natural gas volume purchased, p ele (t k ) and p gas (t k ) is the decision time t k External electricity and natural gas prices.

[0256] The incentive cost is the total compensation cost given by the integrated energy system operator to users who participate in the integrated demand response, which can be expressed as:

[0257] C insp (t k )=C cut (t k )+C ac (t k )

[0258] Maintenance cost refers to the maintenance cost of various internal equipment during operation by the integrated energy system operator, which can be expressed as:

[0259]

[0260] Among them, c mc,n and P n (t k ) is the unit power maintenance cost of equipment n and the decision time t k Output of device n, where N is the total number of devices.

[0261] The carbon tax cost is the environmental protection fee levied by the environmental protection department due to the pollution caused by the system during operation, which can be expressed as:

[0262]

[0263] Among them, ω Ctax is the carbon tax coefficient, E gas and E grid is the CO2 emission per unit of natural gas and electricity, η grid The transmission efficiency of the power grid.

[0264] Step 2: Build the deep neural network required for the Twin Delayed Deep Deterministic Policy Gradient Algorithm (TD3):

[0265] TD3 has three independent neural networks: the policy network, value network 1, and value network 2. Each network has its own target network: the target policy network, target value network 1, and target value network 2. The above six neural networks are all fully connected neural networks, including input layers, hidden layers, and output layers. The policy network and target policy network have the same structure, and the value network 1, value network 2, target value network 1, and target value network 2 have the same structure.

[0266] Step 3: Read Figure 4 For the energy management solution of the integrated energy system based on the TD3 algorithm, according to the model established in step 1, the energy management center interacts with the integrated energy system to obtain historical interaction information and store it in the experience pool. TD3 is used to iteratively update and optimize the policy network, value network 1, and value network 2.

[0267] Step 3.1: Initialize learning and decision parameters, including: initializing the number of decision moments K in a day; initializing the number of learning steps STEP; initializing the number of learning steps step = 0; initializing the sample pool capacity and learning sample batch size M and Batch; initializing the discount factor γ and the soft update coefficient τ; initializing the network parameter update cycle Cycle; initializing the neural network, including: initializing the parameters θ of the policy network, value network 1 and value network 2 respectively π 、 and Initialize the parameters θ of the target policy network, target value network 1, and target value network 2 π′ ←θ π 、 and

[0268] Step 3.2: Initialize the decision time k=0, initialize the integrated energy system state, process the system operation data through the energy management center, and store the generated samples in the experience pool:

[0269] Step 3.2.1, the state s k Normalize to get Will Input the current policy network to get the corresponding action π(s k |θ π ), superimpose noise v1 to obtain random action a k , that is: a k =π(s k |θ π )+v1, execute the currently selected action a k , after a decision cycle, the system reaches the next state s k+1 , and observe the running cost r in the process k , get the sample [sk ,a k ,r k ,s k+1 ], and stored in the experience pool after normalization;

[0270] Step 3.2.2: Set k = k + 1. If k < K, return to step 3.2.1. If k = K, proceed to step 3.3.

[0271] Step 3.3: Randomly select a batch of quadruple sample data from the experience pool

[0272] Step 3.4: Obtain the state through the target policy network The target action To improve the robustness of the training process, motion noise is superimposed on the target action. Get random target action Right now:

[0273] Step 3.5: Calculate the state through the value network Next action The corresponding value function and

[0274] Step 3.6, through the target value network, get the state Random target action The corresponding target value function and According to the Bellman equation, the state Next action The corresponding target value function Q target ,Right now:

[0275] Step 3.7, update the network parameters by minimizing the loss function. The loss function It can be expressed as:

[0276] Step 3.8: Set step = step + 1. If step = N l *Cycle,N l ∈Z + , then the policy network is updated through deterministic policy gradient, which can be expressed as:

[0277]

[0278] Parameters θ of target policy network, target value network 1 and target value network 2 π′ 、 and The parameters of the policy network, value network 1, and value network 2 are obtained through soft update and can be expressed as: θ π′ =(1-τ)θ π′ +τθ π 、 If step < STEP, return to step 3.2; if step = STEP, stop learning and the network training is completed.

[0279] Step 4: Implement control of the integrated energy system based on the strategy network obtained in step 3:

[0280] The feasibility of the verification algorithm is verified using the integrated energy system established above. At any decision-making moment in the system operation, the decision-making moment, invitation response amount, and operating status of the integrated energy system are normalized and input into the strategy network of TD3. After forward propagation, the action information output by the strategy network is denormalized to obtain the actions that the system can take at the current decision moment. After the integrated energy system repeatedly performs the above operations within a scheduling cycle, the completion status of the system's power demand response and the energy sales revenue, response revenue, energy purchase cost, incentive cost, maintenance cost, and carbon tax cost obtained are observed.

[0281] It should be noted that, in this document, relational terms such as first and second, etc., are used solely to distinguish one entity or operation from another, and do not necessarily require or imply any actual relationship or order between these entities or operations. Furthermore, the terms "include," "comprise," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. Without further limitation, elements defined by the phrase "include..." or "comprising..." do not exclude the presence of additional elements in the process, method, article, or terminal device comprising the elements. Furthermore, in this document, "greater than," "less than," "exceeding," etc., are understood to exclude the number itself; "above," "below," "within," etc., are understood to include the number itself.

[0282] Although the above embodiments have been described, those skilled in the art may make additional changes and modifications to these embodiments once they know the basic creative concepts. Therefore, the above descriptions are merely embodiments of the present invention and do not limit the scope of patent protection of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention's description and drawings, or directly or indirectly applied in other related technical fields, are also included in the scope of patent protection of the present invention.

Claims

1. An energy management method for an integrated energy system under demand response, comprising an integrated energy system, wherein the integrated energy system comprises an energy management center, energy conversion equipment, energy storage equipment and multiple types of users, wherein the energy conversion equipment comprises a gas turbine, a waste heat boiler, a gas boiler, an absorption chiller, an electric heat pump, an electric refrigeration unit and a photovoltaic power generation device, wherein the energy storage equipment comprises a battery and a heat storage tank, wherein the multiple types of users comprise public service users and commercial users, wherein the user loads include three energy forms: electric load, heat load and cooling load, wherein the energy management center is an integration center for energy and information, externally accepting power suppliers and natural gas suppliers, and being able to obtain information on electricity prices, natural gas prices and demand response plans issued by power companies, and control the purchase amount of electricity and natural gas energy, and internally issuing response invitations to multiple types of users to adjust the inherent energy consumption behavior of users and the production plan of energy equipment within the system, characterized in that The energy management method specifically includes the following steps: Step 1: The integrated energy system participates in power demand response with a dispatch cycle of one day. The overall goal of its optimized operation is to maximize the total profit of the system operation in one day on the basis of achieving the power demand response target. An operation optimization model for the integrated energy system participating in power demand response is established: Step 1.1: Establish a model for the power demand response plan, PV output randomness, outdoor temperature randomness, and user heating and cooling load randomness. Step 1.2: Establish a scheduling model for energy conversion equipment and energy storage equipment in an integrated energy system; Step 1.3: Establish a multi-class user response characteristic model within the integrated energy system; Step 1.4: Establish a dispatch optimization model for the integrated energy system participating in power demand response; Step 2: Build the deep neural network required for the double-delayed deep deterministic policy gradient algorithm: The dual-delay deep deterministic policy gradient algorithm has three independent neural networks: the policy network, value network 1, and value network 2. Each network has its own target network: the target policy network, target value network 1, and target value network 2. The six neural networks are all fully connected neural networks, including an input layer, a hidden layer, and an output layer. The policy network and the target policy network have the same structure, and the value network 1, value network 2, target value network 1, and target value network 2 have the same structure. Step 3: Based on the model established in step 1, the energy management center interacts with the integrated energy system to obtain historical interaction information and store it in the experience pool. TD3 is used to iteratively update and optimize the policy network, value network 1, and value network 2. Step 4: Implement control of the integrated energy system based on the strategy network obtained in step 3: At any decision moment in the operation of the system, the decision moment, invitation response amount and operating status information of the integrated energy system are normalized and input into the policy network of the double-delay deep deterministic policy gradient algorithm. After forward propagation, the action information output by the policy network is denormalized to obtain the actions that the system can take at the current decision moment. After the integrated energy system repeatedly performs the above operations within a scheduling cycle, the completion status of the system's power demand response and the energy sales revenue, response revenue, energy purchase cost, incentive cost, maintenance cost and carbon tax cost obtained are observed.

2. The energy management method for an integrated energy system under demand response according to claim 1, characterized in that: The power demand response plan, photovoltaic output randomness, outdoor temperature randomness, and user heating and cooling load randomness model described in step 1.1 are specifically as follows: Determine the amount of demand response required to participate in electricity demand response: Determine the decision time t of the integrated energy system in a scheduling cycle k The agreed response quantity P′ of the power demand response peak (t k ), and the demand response target of the integrated energy system at each decision moment in a scheduling cycle is obtained: {P′ peak (t1),P′ peak (t1),...,P′ peak (t k ),...,P′ peak (t K )} Photovoltaic output randomness: Photovoltaic output error ΔP pv Subject to the mean μ pv The variance is σ pv Normal distribution, that is, ΔP pv ~N(μ pv ,σ pv 2 ), its probability density function can be expressed as: If the decision time t k The predicted value of photovoltaic output is P pv,0 (t k ), after adding random errors, the actual value of photovoltaic output P can be obtained pv (t k )for: P pv (t k )=P pv,0 (t k )+ΔP pv Temperature randomness: Temperature error ΔT temp Subject to the mean μ temp The variance is σ temp Normal distribution, that is, ΔT temp ~N(μ temp ,σ temp 2 ), its probability density function can be expressed as: If the decision time t k The predicted temperature is T temp,0 (t k ), after adding random errors, the actual temperature value T temp (t k )for: T temp (t k )=T temp,0 (t k )+ΔT temp Electric load randomness: Error of electrical load ΔL ele Subject to the mean μ ele The variance is σ ele Normal distribution, that is, ΔL ele ~N(μ ele ,σ ele 2 ), its probability density function can be expressed as: If the decision time t k The predicted value of the electric load is L ele,0 (t k ), after adding random errors, the actual value of the load can be obtained L ele (t k )for: L ele (t k )=L ele,0 (t k )+ΔL ele Heat load randomness: Thermal load error ΔL hot Subject to the mean μ hot The variance is σ hot Normal distribution, that is, ΔL hot ~N(μ hot ,σ hot 2 ), its probability density function can be expressed as: If the decision time t k The predicted value of heat load is L hot,0 (t k ), after adding random errors, the actual value of the heat load can be obtained L hot (t k )for: L hot (t k )=L hot,0 (t k )+ΔL hot Cooling load randomness: Error of cooling load ΔL cold Subject to the mean μ cold The variance is σ cold Normal distribution, that is, ΔL cold ~N(μ cold ,σ cold 2 ), its probability density function can be expressed as: If the decision time t k The predicted value of cooling load is L cold,0 (t k ), after adding random errors, the actual value of cooling load L cold (t k )for: L cold (t k )=L cold,0 (t k )+ΔL cold 。 3. The energy management method for an integrated energy system under demand response according to claim 2, characterized in that: The integrated energy system energy conversion equipment and energy storage equipment scheduling model of step 1.2 has the following specific steps: Gas turbine dispatch process: The gas turbine consumes natural gas to provide electricity and heat. The decision time t k The electric power output of the gas turbine is calculated as follows: P gt (t k )=V gt (t k )H ng η gt Among them, P gt (t k ) and V gt (t k ) is the decision time t k The electrical power output of the gas turbine and the natural gas consumed; H ng is the calorific value of natural gas; η gt is the power generation efficiency of the gas turbine; The ratio of the gas turbine output thermal power to electrical power is the thermal-electric ratio b, which can be expressed as: Among them, Q gt (t k ) is the decision time t k The heating power of the gas turbine; Waste heat boiler scheduling process: Waste heat boilers can improve energy utilization efficiency and provide thermal energy to users by absorbing waste heat from high-temperature flue gas discharged by gas turbines. Its mathematical model is: Q hb (t k )=Q hb,0 (t k )η hb Among them, Q hb (t k ) and Q hb,0 (t k )Decision time t k Thermal power absorbed and output by waste heat boiler; η hb is the waste heat recovery efficiency; Gas boiler scheduling process: Gas boiler consumes natural gas to provide heat energy. At the decision time t k The output thermal power is calculated as follows: Q gb (t k )=V gb (t k )H ng η gb Among them, Q gb (t k ) and V gb (t k ) is the decision time t k The thermal power output of the gas boiler and the natural gas consumed; η gb is the heat production efficiency of the gas boiler; Absorption chiller scheduling process: The absorption chiller provides cooling power by absorbing heat power. k The output cooling power is calculated as follows: H ar (t k )=Q ar (t k )η ar Among them, H ar (t k ) and Q ar (t k ) is the decision time t k The cooling power output and heating power absorbed by the absorption chiller; η ar is the refrigeration efficiency of the absorption chiller; Electric heat pump scheduling process: The electric heat pump converts low-quality heat energy into high-quality heat energy by consuming electricity. k The thermal power output of the electric heat pump is calculated as follows: Q eb (t k )=P eb (t k )η eb Among them, Q eb (t k ) and P eb (t k ) is the decision time t k The thermal power output and electrical power consumed by the electric heat pump; η eb is the heating efficiency of the electric heat pump; Electric refrigeration unit scheduling process: The electric refrigeration unit provides cooling power by consuming electric power. The decision time t k The cooling power output of the electric refrigeration unit is calculated as follows: H er (t k )=P er (t k )η er Among them, H er (t k ) and P er (t k ) is the decision time t k The cooling power output and electrical power consumed by the electric refrigeration unit; η er is the efficiency of the electric refrigeration unit; Photovoltaic power generation device scheduling process: Photovoltaic power generation devices use solar energy to generate electricity, and their power generation can be expressed as: Among them, P pv (t k ) and G light (t k ) is the decision time t k The power generation of the photovoltaic power generation device and the light intensity at this time; p stc and G stc is the power generation capacity and corresponding light intensity of the photovoltaic power generation device under standard conditions; Battery scheduling process: Define the decision time t k The state of charge of the battery is SOC es (t k ), which indicates the percentage of the battery's remaining capacity to its rated capacity. Taking into account the battery's power dissipation and charging and discharging during use, the battery's dynamic charging and discharging process is described as follows: SOCIETY es (t k )=(1-σ es )SOC es (t k -1)Δt-η es P es (t k )Δt / V es Where Δt is the time difference between two decision moments, σ es is the battery energy loss rate; P es (t k ) is the decision time t k Battery charge and discharge power, positive value represents discharge, negative value represents charge, 0 represents idle state; V es is the capacity of the battery; η es is the battery charge and discharge coefficient, which can be expressed as: in, and The charging and discharging efficiency of the battery; Thermal storage tank scheduling process: Based on the above description of the battery state of charge, the decision time t is defined as k The thermal energy storage state of the heat storage tank is SOC hs (t k ), the dynamic storage and discharge process of the heat storage tank is described as follows: SOCIETY hs (t k )=(1-σ hs )SOC hs (t k -1)Δt-η hs Q hs (t k )Δt / V hs Among them, σ hs is the energy loss rate of the heat storage tank; Q hs (t k ) is the decision time t k The heat storage tank heat storage and release power, positive value represents heat release, negative value represents heat storage, 0 represents idle state; V hs is the capacity of the heat storage tank; η hs is the heat storage and release coefficient of the heat storage tank, which can be expressed as: in, and is the heat storage and release efficiency of the heat storage tank.

4. The energy management method for an integrated energy system under demand response according to claim 3 is characterized in that: The specific steps of the multi-class user response characteristic model within the integrated energy system in step 1.3 are as follows: Public service user response characteristic model: Public service users refer to office buildings and hospital buildings that serve residents' production and life. They have a large amount of non-essential loads, which can be reduced when necessary. The load reduction model for public service users is as follows: in, and P cut (t k ) is the decision time t k The original power of the load that can be reduced and the power after reduction, α cut (t k ) is the user decision time t k Reduction ratio of load that can be reduced, ε cut- and ε cut+ is the critical compensation price when the user is willing to reduce load and reaches the upper limit of reduction capacity, a and b are load reduction coefficients; The integrated energy system operator issues a subsidy price to the user, and the user reduces the load that can be reduced according to the subsidy price. The user makes a decision at the decision time t k Compensation amount C obtained after load reduction cut (t k ) can be expressed as: Business User Response Characteristic Model: Commercial users include large supermarkets and shopping malls, which have a large amount of air conditioning load. The modeling of air conditioning load includes the thermodynamic modeling of the building to which the air conditioning is located and the electric heat / cooling conversion of the air conditioning. The thermodynamic model of the building to which the air conditioning is located usually adopts an equivalent thermal parameter model based on circuit simulation. The discrete form of the differential equation expression of the first-order equivalent thermal parameter model is: Among them, T a (t k ) and T o (t k ) is the decision time t k Indoor and outdoor air temperature, R a and C a is the equivalent thermal resistance and equivalent heat capacity of indoor air, P ac (t k ) and H ac (t k ) is the decision time t k Air conditioning power and cooling power, η ac For air conditioning cooling efficiency; Assume that the user sets the temperature to the most comfortable temperature T when not participating in the response a,fit , we can get the decision time t through the above formula k Air conditioning power consumption P when the user does not participate in the response ac,fit (t k )for: The indoor temperature can be maintained within the acceptable range for human body[T a,min ,T a,max ], the air conditioner is allowed to participate in the dispatching, and the integrated energy system operator signs a contract with the user to agree on the adjustment of the compensation price ε for the air conditioner power consumption. ac , adjust the power consumption of the air conditioner when necessary, and the user makes the decision at the time t k The amount of compensation received is: C ac (t k )=ε ac |P ac (t k )-P ac,fit (t k )|Δt.

5. The energy management method for an integrated energy system under demand response according to claim 4 is characterized in that: The integrated energy system in step 1.4 participates in the dispatch optimization model of power demand response, and the specific steps are as follows: After receiving the peak load regulation order from the upper power grid, the integrated energy system operator formulates the operation plan of each device and issues dispatch instructions to users to achieve the response target and maximize its own profit R, while ensuring the safe and stable operation of the system. The objective function is expressed as follows: Among them, I sell (t k ), I resp (t k ), C buy (t k ), C insp (t k ), C mc (t k ), C Ctax (t k ) are respectively at the decision time t k The energy sales revenue, response revenue, energy purchase cost, incentive cost, maintenance cost and carbon tax cost of the integrated energy system operator; K is the total number of decision moments in the entire dispatch cycle; Energy sales revenue is the sum of the revenue from electricity sales, heat sales, and cooling sales to users by the integrated energy system operator, which can be expressed as: I sell (t k )=[p ele (t k )L ele (t k )+p hot (t k )L hot (t k )+p cold (t k )L cold (t k )]Δt Among them, L ele (t k ), L hot (t k ) and L cold (t k ) are the decision time t k The user's electrical load, heating load and cooling load power; p ele (t k ), p hot (t k ) and p cold (t k ) are the decision time t k The prices of electricity, heat and cooling sold to users; The response benefit is the economic compensation obtained by the integrated energy system after responding to the power demand of the grid, which can be expressed as: I resp (t k )=γ(t k )ε peak (t k )|P peak (t k )|Δt Among them, γ(t k ) and ε peak (t k ) is the decision time t k The load response rate coefficient and compensation price of power demand response, γ(t k ) and decision time t k The load response rate is related to P peak (t k ) is the decision time t k The actual response power of the integrated energy system, when positive, represents valley filling, and when negative, represents peak shaving; The energy purchase cost is the sum of the costs of electricity and natural gas purchased by the integrated energy system operator from the power grid and natural gas grid, which can be expressed as: C buy (t k )=p ele (t k )P grid (t k )Δt+p gas (t k )V gas (t k ) Among them, P grid (t k ) and V gas (t k ) is the system at the decision time t k Electric power and natural gas volume purchased, p ele (t k ) and p gas (t k ) is the decision time t k external electricity and natural gas prices; The incentive cost is the total compensation cost given by the integrated energy system operator to users who participate in the integrated demand response, which can be expressed as: C insp (t k )=C cut (t k )+C ac (t k ) Maintenance cost refers to the maintenance cost of various internal equipment during operation by the integrated energy system operator, which can be expressed as: Among them, c mc,n and P n (t k ) is the unit power maintenance cost of equipment n and the decision time t k The output of device n, where N is the total number of devices; The carbon tax cost is the environmental protection fee levied by the environmental protection department due to the pollution caused by the system during operation, which can be expressed as: Among them, ω Ctax is the carbon tax coefficient, E gas and E grid is the CO2 emission per unit of natural gas and electricity, η grid The transmission efficiency of the power grid.

6. The energy management method for an integrated energy system under demand response according to claim 1, characterized in that: The step 3 uses TD3 to implement iterative update and optimization of the policy network, value network 1 and value network 2. The specific steps are: Step 3.1: Initialize learning and decision parameters, including: initializing the number of decision moments K in a day; initializing the number of learning steps STEP; initializing the number of learning steps step = 0; initializing the sample pool capacity and learning sample batch size M and Batch; initializing the discount factor γ and the soft update coefficient τ; initializing the network parameter update cycle Cycle; initializing the neural network, including: initializing the parameters θ of the policy network, value network 1 and value network 2 respectively π 、 and Initialize the parameters θ of the target policy network, target value network 1, and target value network 2 π’ ←θ π 、 and Step 3.2: Initialize the decision time k = 0, initialize the integrated energy system state, process the system operation data through the energy management center, and store the generated samples in the experience pool: Step 3.2.1, the state s k Normalize to get Will Input the current policy network to get the corresponding action π(s k |θ π ), superimpose noise v1 to obtain random action a k , that is: a k =π(s k |θ π )+v1, execute the currently selected action a k , after a decision cycle, the system reaches the next state s k+1 , and observe the running cost r in the process k , get the sample [s k ,a k ,r k ,s k+1 ], and stored in the experience pool after normalization; Step 3.2.2: Set k = k + 1. If k < K, return to step 3.2.

1. If k = K, proceed to step 3.

3. Step 3.3: Randomly select a batch of quadruple sample data from the experience pool Step 3.4: Obtain the state through the target policy network The target action To improve the robustness of the training process, motion noise is superimposed on the target action. Get random target action Right now: Step 3.5: Calculate the state through the value network Next action The corresponding value function and Step 3.6, through the target value network, get the state Random target action The corresponding target value function and According to the Bellman equation, the state Next action The corresponding target value function Q target ,Right now: Step 3.7, update the network parameters by minimizing the loss function. The loss function It can be expressed as: Step 3.8: Set step = step + 1. If step = N l *Cycle,N l ∈Z + , then the policy network is updated through deterministic policy gradient, which can be expressed as: Parameters θ of target policy network, target value network 1 and target value network 2 π’ 、 and The parameters of the policy network, value network 1, and value network 2 are obtained through soft update and can be expressed as: θ π’ =(1-τ)θ π’ +τθ π 、 If step < STEP, return to step 3.2; if step = STEP, stop learning and the network training is completed.

Images