Virtual energy storage modeling and optimization control method and system of building heating ventilation air conditioning system

Abstract

The invention discloses a virtual energy storage modeling and optimization control method and system for a building heating ventilation air-conditioning system, and the method comprises the steps: taking a building as a virtual energy storage node, considering all influence factors of indoor temperature changes, building a thermodynamic model of the building heating ventilation air-conditioning system, and carrying out the optimization control of the building heating ventilation air-conditioning system; establishing a virtual energy storage model of the building heating ventilation air conditioning system based on the thermodynamic model and the correction strategy of the room temperature at the next moment; and in consideration of dynamic electricity price and weather information, converting a control problem of the virtual energy storage model into an MDP model, and performing optimization control on the MDP model by adopting a PPO algorithm. According to the method, the thermal comfort of residents can be effectively improved, the building operation cost can be effectively reduced, the regulation and control potential of a building heating ventilation and air conditioning system is fully exerted, and optimized integration and efficient utilization of VES resources are achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of flexibility resource optimization control, and relates to a virtual energy storage modeling and optimization control method and system for a building HVAC system. Background Art

[0002] The energy consumption of building heating, ventilation and air conditioning (HVAC) systems has exceeded 50% of the total energy consumption of buildings, and is increasing year by year. At the same time, building HVAC systems affect indoor thermal comfort and have an important impact on human daily production activities. Due to the unique thermal capacity characteristics of building HVAC systems, they can participate in grid regulation as a flexible resource scheduling method.

[0003] However, the existing modeling methods lack accurate description methods when constructing the thermodynamic model of the building HVAC system, and lack flexibility in the control of building HVAC system equipment, and fail to fully tap the adjustable thermodynamic resources in the building HVAC system. Therefore, it is urgent to explore new technical means to improve the operating efficiency and stability of the power grid. Summary of the invention

[0004] To address the deficiencies in the prior art, the present invention provides a virtual energy storage modeling and optimization control method and system for a building HVAC system, which can accurately model different types of building HVAC systems and achieve flexible and efficient control, effectively improve the thermal comfort of occupants and reduce building operating costs, fully tap the potential of buildings as virtual energy storage resources in power grid regulation, optimize and integrate virtual energy storage (VES) resources, improve resource utilization efficiency, and ensure the stability and operating efficiency of the power grid under different load conditions.

[0005] The present invention adopts the following technical solution.

[0006] The first aspect of the present invention provides a virtual energy storage modeling and optimization control method for a building HVAC system, comprising:

[0007] Taking the building as a virtual energy storage node, considering all the influencing factors of indoor temperature changes, a thermodynamic model of the building HVAC system is established, and based on the thermodynamic model and the correction strategy of the room temperature at the next moment, a virtual energy storage model of the building HVAC system is established;

[0008] Taking into account dynamic electricity prices and weather information, the control problem of the virtual energy storage model is converted into an MDP model, and the PPO algorithm is used to optimize the control of the MDP model.

[0009] Preferably, all influencing factors of the indoor temperature change include the wall heat transfer Q wall , window heat transfer Q win 、Solar radiation heat transfer Q sol , ventilation system heat transfer Q ven , heat pump heat transfer Q jc , Radiant heat transfer of electrical equipment Q app And the human body radiation heat transfer Q peo , as follows:

[0010]

[0011] Where N j Indicates the total number of exterior walls; U wall Represents the equivalent heat transfer coefficient of the wall; A wall,j Indicates the wall area of ​​each wall; and Respectively represent the outdoor temperature and room temperature at the current time t;

[0012]

[0013] Where N p Indicates the total number of windows; U win A represents the equivalent heat transfer coefficient of the window; win,p represents the area of ​​the face window p;

[0014]

[0015] In the formula, β win,p represents the shading coefficient of window p; represents the solar radiation intensity at window p;

[0016]

[0017] Where ρ represents the air density; C ρ represents the specific heat capacity of air; Indicates the ventilation volume of the ventilation system; q l Indicates the air leakage rate of the ventilation system; η HR is the heat recovery rate of the ventilation system;

[0018]

[0019] Where η HC It represents the power coefficient of the heat pump system; Indicates the power of the heat pump;

[0020]

[0021] In the formula, Indicates the utilization rate of electrical equipment; Appl per Indicates the power of electrical equipment per square meter; A room Represents the total area of ​​the house; ε appl Indicates the heating power of the appliance;

[0022]

[0023] In the formula, Indicates the occupancy rate of people in the room at a certain moment; Body per Represents the heat generation power of the human body; it is determined by the metabolic rate of the human body; Num represents the total number of people in the building.

[0024] Preferably, the building is taken as a virtual energy storage node, and all influencing factors of indoor temperature change are considered. The thermodynamic model of the building HVAC system is established as follows:

[0025]

[0026] Where ρ represents the air density; C ρ Represents the specific heat capacity of air; V room Indicates the volume of the house; represents the room temperature at the current time t;

[0027] Q wall Indicates the heat transfer of the wall; Q win Indicates the heat transfer of the window; Q sol Indicates the amount of heat transferred by solar radiation; Q ven Indicates the heat transfer of the ventilation system; Q hc Indicates the heat transfer of the heat pump; Q app Indicates the amount of heat transferred by radiation from electrical equipment; Q peo Indicates the amount of heat transferred by human radiation;

[0028] represents the room temperature of the house at the next moment t; Δt represents the time interval.

[0029] Preferably, the correction strategy corrects the room temperature at the next moment based on a thermodynamic model, divides the controlled time period into time intervals, and obtains a virtual energy storage model of the building HVAC system as shown below:

[0030]

[0031] Wherein, N represents the number of time intervals divided within the controlled time period, i is the serial number of the divided time interval, and Δt′ is the divided time interval.

[0032] Preferably, the dynamic electricity price and weather information are considered, and the control problem of the virtual energy storage model is converted into an MDP model, and the state information of the MDP model is:

[0033]

[0034] in, represents the room temperature at the current time t;

[0035] represents the outdoor temperature at time t, t+1, and t+6;

[0036] represents the solar radiation intensity at window p at time t, t+1, and t+6;

[0037] T represents the time period of control;

[0038] Price t 、Price t+1 Represents the electricity price at time t and t+1.

[0039] Preferably, the control action of the MDP model is:

[0040]

[0041] in, Indicates the power of the heat pump; Indicates the ventilation volume of the ventilation system.

[0042] Preferably, the optimization objective of the MDP model is:

[0043]

[0044] Among them, min means minimization; Abs() is the absolute value function;

[0045] w1 and w2 are the weights of electricity cost and thermal comfort cost respectively;

[0046] Δt represents the time interval; σ is the thermal comfort cost coefficient; are the maximum and minimum values ​​at room temperature.

[0047] Preferably, the reward function of the MDP model is:

[0048]

[0049] Among them, R t is the reward function at the current time t.

[0050] Preferably, the PPO algorithm uses an Actor-Critic network structure to optimize and control the MDP model.

[0051] The second aspect of the present invention provides a virtual energy storage modeling and optimization control system for a building HVAC system, comprising:

[0052] The model building module is used to take the building as a virtual energy storage node, consider all the influencing factors of indoor temperature changes, establish a thermodynamic model of the building HVAC system, and establish a virtual energy storage model of the building HVAC system based on the thermodynamic model and the correction strategy of the room temperature at the next moment;

[0053] The optimization control module is used to consider dynamic electricity prices and weather information, convert the control problem of the virtual energy storage model into an MDP model, and use a PPO algorithm to optimize the control of the MDP model.

[0054] A third aspect of the present invention provides a terminal, comprising a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method.

[0055] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method when executed by a processor.

[0056] Compared with the prior art, the beneficial effects of the present invention include at least:

[0057] Based on the spatiotemporal dynamic change characteristics of building energy consumption and load, the present invention takes the building as a VES node, comprehensively considers weather factors, thermal comfort of personnel and dynamic electricity prices, constructs the building HVAC system control as a virtual energy storage model through thermodynamic model construction, and combines the correction strategy of room temperature at the next moment. The control problem of the virtual energy storage model is converted into an MDP model, and the PPO algorithm is used to optimize the MDP model. The proposed correction strategy, virtual energy storage model of the building HVAC system, and MDP model can more scientifically reflect the changes in indoor temperature, and can significantly reduce the volatility of indoor temperature during simulation, which can reduce the operating cost while ensuring indoor thermal comfort and dispatch and balance the grid load.

[0058] The present invention uses deep reinforcement learning technology to control the virtual energy storage model. Compared with other control methods such as rule control and model predictive control, deep reinforcement learning technology has the advantages of strong robustness and wide application range. It can optimize and adjust the building load efficiently and flexibly in multiple scenarios. It can reduce operating costs and flexibly dispatch and balance the grid load while ensuring indoor thermal comfort, provide strong support for the safe and stable operation of the power grid, and realize precise control and efficient management of building VES resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is composed of interactive environment of intelligent building;

[0060] Figure 2 It is a deep reinforcement learning algorithm framework based on the PPO algorithm;

[0061] Figure 3 Markov decision process for HVAC control systems;

[0062] Figure 4 It is the Actor network structure;

[0063] Figure 5 The figure is a flowchart of the method of the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. The embodiments described in this application are only embodiments of a part of the present invention, rather than all embodiments. Based on the spirit of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the protection scope of the present invention.

[0065] Embodiment 1 of the present invention proposes a virtual energy storage modeling and optimization control method for a building HVAC system, focusing on the dynamic thermal change characteristics of the building, considering the multiple components of the thermodynamic model of the building HVAC system, establishing a mathematical model for each part, and forming a complete thermodynamic mathematical model of the building HVAC system; forming a dynamic building virtual energy storage model based on the thermodynamic model of the building HVAC system; based on the above virtual energy storage model, considering weather information, personnel information and other related information, using the PPO algorithm based on the MDP process to optimize and control the VES model of the building HVAC system, which can effectively improve the thermal comfort of the occupants and reduce the operating costs of the building, give full play to the control potential of the building HVAC system, and achieve optimized integration and efficient use of VES resources. Figure 5 As shown, the method specifically includes:

[0066] Step 1: Taking the building as a virtual energy storage node, considering all the influencing factors of indoor temperature changes, establishing a thermodynamic model of the building HVAC system, and establishing a virtual energy storage model of the building HVAC system based on the thermodynamic model and the correction strategy of the room temperature at the next moment;

[0067] Furthermore, the rise of virtual energy storage (VES) technology provides a new idea for the integration and management of distributed resources. By aggregating distributed energy storage resources to form a virtual energy storage system, flexible scheduling and balancing of grid loads can be achieved. As a new type of communication infrastructure, the building HVAC system has the potential to be a virtual energy storage resource.

[0068] The present invention specifically analyzes the dynamic changes of the building HVAC system, establishes a thermodynamic model of the building HVAC system based on the thermodynamic changes of the building, and on this basis, establishes a virtual energy storage model of the building HVAC system; the thermodynamic model of the building HVAC system and the virtual energy storage modeling are specifically as follows:

[0069] 1. Thermodynamic model of building HVAC system

[0070] Typical building HVAC systems such as Figure 1 As shown in the figure, there are 7 factors that affect indoor temperature changes, namely wall heat transfer, window heat transfer, solar radiation, ventilation system heat transfer, heat pump, electrical appliance radiation, and human body heat. The above 7 influencing factors need to be modeled:

[0071] For wall heat transfer, the formula is as follows (1). In this formula, N j Refers to the total number of external walls. Usually, the roof can also be considered as an external wall. wall Refers to the equivalent heat transfer coefficient of the wall. Different values ​​need to be selected for different wall materials. wall,j Refers to the wall area of ​​each wall. and Refers to the temperature inside and outside the wall respectively. The temperature difference will cause heat transfer. wall Refers to the sum of all wall heat transfer powers.

[0072]

[0073] For window heat transfer, the formula is as follows (2). In this formula, N p Refers to the total number of windows. win Refers to the equivalent heat transfer coefficient of the window. win,p Refers to the area of ​​each window. The heat transfer of windows is also driven by the temperature difference between indoor and outdoor, Q win It refers to the total heat transfer of all windows.

[0074]

[0075] For solar radiation, the formula is as follows (3). Since solar radiation is projected into the room through windows, the solar radiation intensity of each window needs to be considered in the calculation. In this formula, βwin,p It refers to the shading coefficient of each window, which is used to calculate the transmittance of solar radiation and the impact of different window angles. Refers to the solar radiation intensity at each window. sol Refers to the sum of the power of solar radiation projected by all windows.

[0076]

[0077] For heat transfer in ventilation systems, the formula is as follows (4). In this formula, ρ refers to the air density, C ρ Refers to the specific heat capacity of air. These two items describe the physical properties of air. Refers to the ventilation volume of the ventilation system. l Refers to the air leakage rate of the ventilation system, η HR is the heat recovery rate of the ventilation system.

[0078]

[0079] For heat pump heat transfer, the formula is as follows (5). In this formula, η HC Refers to the power coefficient of the heat pump system. Refers to the power of the heat pump, which can be positive or negative. A positive value means that the heat pump system transfers heat to the room, and a negative value means that the heat pump system cools the room.

[0080]

[0081] For the radiation heat transfer of electrical equipment, the formula is as follows (6). In this formula, Refers to the utilization rate of electrical equipment, that is, the proportion of all electrical equipment in use at a certain moment. per Refers to the power of electrical equipment per square meter. room Refers to the total area of ​​the house. appl Refers to the heat output of the appliance. The amount of heat transferred from the appliance to the room is expressed in Q app To express.

[0082]

[0083] For human body radiation heat transfer, the formula is as follows (7). In this formula, Refers to the occupancy rate of people in a room at a certain moment, which changes with time. per Refers to the heat generation power of the human body, which is determined by the metabolic rate of the human body. Num refers to the total number of people in the building. peo Refers to the sum of all heat generating powers.

[0084]

[0085] According to the above seven factors that affect indoor temperature changes, the dynamic changes of indoor temperature can be summarized and summarized, which is the thermodynamic model of the building HVAC system. The formula is as follows:

[0086]

[0087] Where: Q wall Indicates the heat transfer of the wall; Q win Indicates the heat transfer of the window; Q sol Indicates the amount of heat transferred by solar radiation; Q ven Indicates the heat transfer of the ventilation system; Q hc Indicates the heat transfer of the heat pump; Q app Indicates the heat transfer of electrical equipment; Q peo Indicates the amount of heat transferred by personnel;

[0088] In this formula, V room Refers to the volume of the house. Refers to the current temperature of the house. In this formula, the temperature is calculated using the differential method, that is,

[0089] When calculating the temperature at the next moment, use formula (9) to convert the differential into an integral form, as shown below:

[0090]

[0091] The thermodynamic model of the building HVAC system directly affects the effect of the control method. Therefore, the present invention models all the influencing factors of the building HVAC system and establishes a complete indoor temperature change model based on this, thereby establishing a virtual energy storage model of the building HVAC system for optimized control.

[0092] 2. Considering the dynamic changes of the building HVAC system, the room temperature calculation formula at the next moment is revised to establish a virtual energy storage model of the building HVAC system:

[0093] The temperature formula of the building HVAC system corresponding to the next moment is as follows:

[0094]

[0095] In this formula, the selection of Δt is crucial to calculate a reasonable and correct indoor temperature. The research using the above model ignores the characteristic that the temperature in the building does not change much in a short period of time.

[0096] During calculation, Δt is often set to 1h. Although this meets the control time period requirement, in actual processes, the temperature simulated in this way often has extremely strong fluctuations and cannot meet the needs of correctly simulating ambient temperature changes.

[0097] In order to solve the problem of indoor temperature fluctuation and drastic change, the present invention modifies the room temperature calculation formula at the next moment. The modification strategy is to divide the control time period into time intervals, and the modified virtual energy storage model formula of the building HVAC system can be obtained as shown in (11):

[0098]

[0099] In formula (11), N refers to the number of time intervals used for calculation within the control time period.

[0100] For example, when the control time period is 1 hour, the calculation time interval used in the present invention is Δt=5 minutes, and at this time, N=12.

[0101] The heat transfer of the surface wall at the current moment; The amount of heat transferred by the table window at the current moment; Surface solar radiation heat transfer; The heat transfer of the surface ventilation system at the current moment; The heat transfer of the surface heat pump system at the current moment; The heat transfer of the personnel at the current moment; The heat transfer of the electrical equipment at the current moment.

[0102] Compared with the original formula, the revised formula can more scientifically reflect the changes in indoor temperature and significantly reduce the volatility of indoor temperature during simulation. According to the above method, the building HVAC system can be converted into a virtual energy storage model.

[0103] Step 2: Considering dynamic electricity prices and weather information, the control problem of the virtual energy storage model is converted into an MDP model, and the PPO algorithm is used to optimize the control of the MDP model.

[0104] Further preferably, the operating load of the building HVAC system is related to the indoor temperature and thermal comfort, which will affect the overall energy consumption of the building. By accurately modeling the building HVAC system and flexibly controlling the power consumption of the building HVAC system, it is possible to establish a virtual energy storage model of the building HVAC system and tap its adjustable potential, which can provide additional energy support for the power grid and participate in the load balancing of the power grid. However, traditional control methods have obvious deficiencies in computing speed and control performance, which directly affect the utilization of virtual energy storage in building HVAC systems. Deep reinforcement learning technology has strong flexibility and excellent control performance and can adapt to a variety of systems. The present invention is based on the virtual energy storage model of the building HVAC system, considers dynamic electricity prices and weather information, and uses a deep reinforcement learning control method to achieve optimal control. Specifically, the established virtual energy storage model of the building HVAC system is converted into a Markov decision process (MDP), and the deep reinforcement learning algorithm proximal policy optimization (PPO) is used to achieve optimal control.

[0105] 1. Transform the control problem into MDP

[0106] For intelligent control of HVAC systems, the actual control process can be regarded as a Markov decision process. The control system itself can be regarded as an intelligent agent, thus forming a Markov decision process.

[0107] 2. PPO algorithm control based on MDP process

[0108] The present invention takes into account the information at the current moment and the information at the future moment, and uses the PPO algorithm to optimize the established MDP process. The basic process is as follows: Figure 2 shown.

[0109] The specific steps are as follows:

[0110] 1): Establish the MDP model of the building HVAC virtual energy storage control system as follows Figure 3 Its status information includes the current status information, such as the solar radiation intensity, outdoor temperature, time, dynamic electricity price and other information at this moment, and also includes the future status information, as shown in formula (12):

[0111]

[0112] The control action is a combination of the ventilation volume of the ventilation system and the power of the heat pump, which is a continuous interval control as shown in formula (13):

[0113]

[0114] The optimization goal of this control system is to minimize the electricity price while ensuring thermal comfort. Therefore, its formula is shown in (14):

[0115]

[0116] In this formula, min represents the minimization of system cost, which means that the system can minimize the electricity cost and the user's thermal comfort cost; w1 and w2 are the weights of electricity cost and thermal comfort cost respectively; Heat pump power The absolute value of heat pump power, positive heat pump power represents indoor heating, negative heat pump power represents cooling; T refers to the controlled time period, price t Refers to the real-time electricity price information, and σ is the thermal comfort cost coefficient. There are two parameters in this formula, w1 and w2, which control the weight of these two types of costs. According to this optimization goal, the reward function of MDP is defined as follows:

[0117]

[0118] Thus, the building HVAC virtual energy storage model control is transformed into an MDP model.

[0119] 2): The PPO algorithm is used to optimize the control of the above MDP model, using the Actor-Critic network structure, whose structure is as follows Figure 4 shown.

[0120] The PPO algorithm uses an Actor-Critic network structure to optimize and control the MDP model, and the optimization makes the total cost the lowest, that is, the electricity cost and the thermal comfort cost are the lowest. Corresponding to formula (14), the lower the total cost, the better the algorithm effect.

[0121] The present invention evaluates two criteria, namely electricity cost and thermal comfort cost.

[0122] The cost of thermal comfort is determined by the PMV index, which is the average air temperature t when the PMV index is minimum and maximum. a As the above

[0123] The general calculation formula of PMV indicator is as follows:

[0124]

[0125] Among them, M refers to the human metabolic rate, and its unit is W / m 2 The human body has different metabolic rates in different situations, and the average metabolic rate of the human body is generally used for calculation. W is the power of the human body doing work externally, and its unit is W / m 2 Different movements require different powers, and the power of work done in buildings can generally be ignored. a is the partial pressure of water vapor, and its unit is Pa.a is the average air temperature in °C. cl is the dressing coefficient. The more clothes you wear, the greater the value. is the mean radiation temperature, in degrees Celsius. In buildings, this value is generally taken to be the same as the average air temperature, i.e.

[0126] The total cost includes electricity price and thermal comfort cost. The lower the total cost, the better the algorithm effect.

[0127] The specific simulation implementation is as follows:

[0128] 1. Parameter settings

[0129] The present invention assumes that the length, width and height of the test house are 10 meters, 10 meters and 3 meters respectively. It is assumed that a family of three lives in the house, and it is assumed that the parents will work from 8 am to 6 pm, and the children will be at home all the time, which means that during the parents' working hours, there is 1 person in the room, and 3 people in the rest of the time. In terms of electrical appliance use, considering the electricity consumption, the usage rate of electrical appliances is 0.2 at night and 0.8 during the day. Other parameter settings in the model are shown in Table 1 below.

[0130] Table 1 Parameters used in modeling

[0131]

[0132]

[0133] In the above table, the window-to-wall ratio ranges from 0.2 to 0.4, and the value used in this paper is 0.2. The solar radiation refractive index is a key factor affecting indoor temperature. Taking into account the influence of window materials and other factors, this paper selects 0.13 as the value for simulation. The roof of the house is treated as an exterior wall. In terms of the power of other indoor electrical equipment, the power of indoor lighting and other electrical appliances is fully considered, which is a total of 23W / m 2 , which means that when the appliance usage rate is 1, 100m 2 In a house, the total power of electrical equipment is 2300W.

[0134] 2. Analysis of example results

[0135] Two typical meteorological months were set up to evaluate the optimization control effect of the deep reinforcement learning method, namely winter and summer typical meteorological months. The comparison results of each parameter are as follows:

[0136] Table 2 Summer experimental results

[0137] method Total cost Average cost Average temperature Exceeding PMV ratio Standard Deviation Average PMV DQN_Future 11471.44 370.05 24.12 67.87％ 4.94 1.17 PPO_noFuture 875.80 28.25 24.38 19.89％ 1.23 0.38 PPO_Future 773.32 24.95 24.56 15.19％ 1.21 0.34

[0138] Table 3 Winter experimental results

[0139] method Total cost Average cost Average temperature Exceeding PMV ratio Standard Deviation Average PMV DQN_Future 9469.25 338.19 26.07 68.45％ 3.03 0.84 PPO_noFuture 4691.62 167.56 24.86 31.99％ 1.85 0.45 PPO_Future 3752.08 134.60 24.45 24.59％ 1.49 0.36

[0140] From the above results, we can know that:

[0141] 1) The method proposed in the present invention achieves the best in all parameters. The summer experimental results are shown in Table 2. In terms of total cost and average cost, the proposed method is far superior to the DQN method of discrete action control, and is also superior to the PPO method that does not utilize future state information. The improvement is 11.68%. In terms of average temperature, the average temperature obtained by the proposed method is closer to the temperature when PMV is 0, which means that the temperature at this time is more comfortable. The proportion of exceeding PMV indicates the proportion of the temperature exceeding the comfortable range specified by PMV. Similarly, the proportion of the proposed method exceeding the specified temperature range is only 15.19%, which is much lower than the other two frameworks. Compared with the PPO method that does not contain future state information, the improvement is 23.63%. The standard deviation refers to the standard deviation of the temperature, which illustrates the stability of the control. The standard deviation of the proposed method in temperature control also reached the lowest.

[0142] 2) Table 3 shows the control performance of the proposed method in winter. As in summer, the proposed method is also better than the other two methods in winter. In terms of cost saving, the improvement ratio is 24.29%. In terms of comfort, the improvement ratio is 30.09%.

[0143] Therefore, when facing different types of building HVAC systems, the deep reinforcement learning algorithm can achieve efficient and flexible control. The method of the present invention can accurately model different types of building HVAC systems and achieve flexible and efficient control.

[0144] Embodiment 2 of the present invention provides a virtual energy storage modeling and optimization control system for a building HVAC system, including:

[0145] The model building module is used to take the building as a virtual energy storage node, consider all the influencing factors of indoor temperature changes, establish a thermodynamic model of the building HVAC system, and establish a virtual energy storage model of the building HVAC system based on the thermodynamic model and the correction strategy of the room temperature at the next moment;

[0146] The optimization control module is used to consider dynamic electricity prices and weather information, convert the control problem of the virtual energy storage model into an MDP model, and use a PPO algorithm to optimize the control of the MDP model.

[0147] Embodiment 3 of the present invention provides a terminal, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method.

[0148] Embodiment 4 of the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method when executed by a processor.

[0149] Compared with the prior art, the beneficial effects of the present invention include at least:

[0150] Based on the spatiotemporal dynamic change characteristics of building energy consumption and load, the present invention takes the building as a VES node, comprehensively considers weather factors, thermal comfort of personnel and dynamic electricity prices, constructs the building HVAC system control as a virtual energy storage model through thermodynamic model construction, and combines the correction strategy of room temperature at the next moment. The control problem of the virtual energy storage model is converted into an MDP model, and the PPO algorithm is used to optimize the MDP model. The proposed correction strategy, virtual energy storage model of the building HVAC system, and MDP model can more scientifically reflect the changes in indoor temperature, and can significantly reduce the volatility of indoor temperature during simulation, which can reduce the operating cost while ensuring indoor thermal comfort and dispatch and balance the grid load.

[0151] The present invention uses deep reinforcement learning technology to control the virtual energy storage model. Compared with other control methods such as rule control and model predictive control, deep reinforcement learning technology has the advantages of strong robustness and wide application range. It can optimize and adjust the building load efficiently and flexibly in multiple scenarios. It can reduce operating costs and flexibly dispatch and balance the grid load while ensuring indoor thermal comfort, provide strong support for the safe and stable operation of the power grid, and realize precise control and efficient management of building VES resources.

[0152] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0153] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0154] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0155] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A virtual energy storage modeling and optimization control method for a building HVAC system, characterized in that: include: Taking the building as a virtual energy storage node, considering all the influencing factors of indoor temperature changes, a thermodynamic model of the building HVAC system is established, and based on the thermodynamic model and the correction strategy of the room temperature at the next moment, a virtual energy storage model of the building HVAC system is established; Taking into account dynamic electricity prices and weather information, the control problem of the virtual energy storage model is converted into an MDP model, and the PPO algorithm is used to optimize the control of the MDP model.

2. The virtual energy storage modeling and optimization control method of a building HVAC system according to claim 1, characterized in that: All influencing factors of indoor temperature change include wall heat transfer Q wall , window heat transfer Q win 、Solar radiation heat transfer Q sol , ventilation system heat transfer Q ven , heat pump heat transfer Q hc , Radiant heat transfer of electrical equipment Q app And the human body radiation heat transfer Q peo , as follows: Where N j Indicates the total number of exterior walls; U wall Represents the equivalent heat transfer coefficient of the wall; A wall,j Indicates the wall area of ​​each wall; and They represent the outdoor temperature and room temperature at the current moment t respectively; Where N p Indicates the total number of windows; U win A represents the equivalent heat transfer coefficient of the window; win,p represents the area of ​​the face window p; In the formula, β win,p represents the shading coefficient of window p; represents the solar radiation intensity at window p; Where ρ represents the air density; C ρ represents the specific heat capacity of air; Indicates the ventilation volume of the ventilation system; q l Indicates the air leakage rate of the ventilation system; η HR is the heat recovery rate of the ventilation system; Where η HC It represents the power coefficient of the heat pump system; Indicates the power of the heat pump; In the formula, Indicates the utilization rate of electrical equipment; Appl per Indicates the power of electrical equipment per square meter; A room represents the total area of ​​the house; appl Indicates the heating power of the appliance; In the formula, Indicates the occupancy rate of people in the room at a certain moment; Body per Represents the heat generation power of the human body; it is determined by the metabolic rate of the human body; Num represents the total number of people in the building.

3. The virtual energy storage modeling and optimization control method of a building HVAC system according to claim 1, characterized in that: Taking the building as a virtual energy storage node and considering all the factors affecting the indoor temperature change, the thermodynamic model of the building HVAC system is established as follows: Where ρ represents the air density; C ρ Represents the specific heat capacity of air; V room Indicates the volume of the house; represents the room temperature at the current time t; Q wall Indicates the heat transfer of the wall; Q win Indicates the heat transfer of the window; Q sol Indicates the amount of heat transferred by solar radiation; Q ven Indicates the heat transfer of the ventilation system; Q hc Indicates the heat transfer of the heat pump; Q app Indicates the amount of heat transferred by radiation from electrical equipment; Q peo Indicates the amount of heat transferred by human radiation; represents the room temperature of the house at the next moment t; Δt represents the time interval.

4. The virtual energy storage modeling and optimization control method of a building HVAC system according to claim 3 is characterized in that: The correction strategy corrects the room temperature at the next moment based on the thermodynamic model, divides the controlled time period into time intervals, and obtains the virtual energy storage model of the building HVAC system as shown below: Wherein, N represents the number of time intervals divided within the controlled time period, i is the serial number of the divided time interval, and Δt′ is the divided time interval.

5. The virtual energy storage modeling and optimization control method of a building HVAC system according to claim 1, characterized in that: The dynamic electricity price and weather information are considered to convert the control problem of the virtual energy storage model into an MDP model, and the state information of the MDP model is: in, represents the room temperature at the current time t; represents the outdoor temperature at time t, t+1, and t+6; represents the solar radiation intensity at window p at time t, t+1, and t+6; T represents the time period of control; Price t 、Price t+1 Represents the electricity price at time t and t+1.

6. A virtual energy storage modeling and optimization control method for a building HVAC system according to claim 5, characterized in that: The control actions of the MDP model are: in, Indicates the power of the heat pump; Indicates the ventilation volume of the ventilation system.

7. A virtual energy storage modeling and optimization control method for a building HVAC system according to claim 6, characterized in that: The optimization goal of the MDP model is: Among them, min means minimization; Abs() is the absolute value function; w1 and w2 are the weights of electricity cost and thermal comfort cost respectively; Δt represents the time interval; σ is the thermal comfort cost coefficient; are the maximum and minimum values ​​at room temperature.

8. A virtual energy storage modeling and optimization control method for a building HVAC system according to claim 7, characterized in that: The reward function of the MDP model is: Among them, R t is the reward function at the current time t.

9. The virtual energy storage modeling and optimization control method of a building HVAC system according to claim 1, characterized in that: The PPO algorithm uses an Actor-Critic network structure to optimize and control the MDP model.

10. A virtual energy storage modeling and optimization control system for a building HVAC system, using the method described in any one of claims 1 to 9, characterized in that: The control system comprises: The model building module is used to take the building as a virtual energy storage node, consider all the influencing factors of indoor temperature changes, establish a thermodynamic model of the building HVAC system, and establish a virtual energy storage model of the building HVAC system based on the thermodynamic model and the correction strategy of the room temperature at the next moment; The optimization control module is used to consider dynamic electricity prices and weather information, convert the control problem of the virtual energy storage model into an MDP model, and use a PPO algorithm to optimize the control of the MDP model.

11. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images