Virtual energy storage modeling and optimization control method and system for building HVAC systems

Abstract

This invention discloses a virtual energy storage modeling and optimization control method and system for building HVAC systems. The method includes: treating the building as a virtual energy storage node, considering all influencing factors of indoor temperature changes, establishing a thermodynamic model of the building HVAC system, and establishing a virtual energy storage model of the building HVAC system based on the thermodynamic model and a correction strategy for the room temperature at the next moment; considering dynamic electricity prices and weather information, transforming the control problem of the virtual energy storage model into an MDP model, and using the PPO algorithm to optimize and control the MDP model. This invention can effectively improve the thermal comfort of residents and reduce building operating costs, fully utilize the regulation potential of building HVAC systems, and achieve optimized integration and efficient utilization of VES resources.

Description

Technical Field

[0001] This invention belongs to the field of flexible resource optimization control technology, and relates to a virtual energy storage modeling and optimization control method and system for building heating, ventilation and air conditioning systems. Background Technology

[0002] Building heating, ventilation, and air conditioning (HVAC) systems now account for over 50% of total building energy consumption, and this figure is increasing year by year. Furthermore, HVAC systems significantly impact indoor thermal comfort and daily human activities. Due to their unique heat capacity characteristics, building HVAC systems can participate in power grid regulation as a flexible resource allocation method.

[0003] However, existing modeling methods lack precise descriptive means when constructing thermodynamic models of building HVAC systems, and are insufficiently flexible in controlling building HVAC system equipment, failing to fully exploit the adjustable thermodynamic resources within the building HVAC system. Therefore, there is an urgent need to explore new technical means to improve the operating efficiency and stability of the power grid. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a virtual energy storage modeling and optimized control method and system for building HVAC systems. This method can accurately model different types of building HVAC systems and achieve flexible and efficient control, effectively improving the thermal comfort of residents and reducing building operating costs. It also fully explores the potential of buildings as virtual energy storage resources in grid regulation, optimizes and integrates virtual energy storage (VES) resources, improves resource utilization efficiency, and ensures grid stability and operational efficiency under different load conditions.

[0005] The present invention adopts the following technical solution.

[0006] The first aspect of this invention proposes a virtual energy storage modeling and optimization control method for building heating, ventilation, and air conditioning systems, comprising:

[0007] The building is used as a virtual energy storage node. All factors affecting indoor temperature changes are considered. A thermodynamic model of the building's HVAC system is established. A virtual energy storage model of the building's HVAC system is established based on the thermodynamic model and the correction strategy for the room temperature at the next moment.

[0008] Considering dynamic electricity prices and weather information, the control problem of the virtual energy storage model is transformed into an MDP model, and the PPO algorithm is used to optimize the control of the MDP model.

[0009] Preferably, all factors influencing the indoor temperature change include the heat transfer of the walls, Q. wall Window heat transfer Q win Solar radiation heat transfer Q sol Heat transfer Q of ventilation system ven Heat pump heat transfer Q jc Radiated heat transfer Q of electrical equipment app and the heat transfer of the human body by radiation Q peo The details are as follows:

[0010]

[0011] In the formula, N j Indicates the total number of exterior walls; U wall A represents the equivalent heat transfer coefficient of the wall; wall,j This indicates the area of ​​each wall. and Let these represent the outdoor temperature and room temperature at the current time t, respectively.

[0012]

[0013] In the formula, N p Indicates the total number of windows; U win Indicates the equivalent heat transfer coefficient of the window; A win,p This represents the area of ​​window p.

[0014]

[0015] In the formula, β win,p This represents the shading coefficient of window p; This represents the intensity of solar radiation at window p;

[0016]

[0017] In the formula, ρ represents air density; C ρ This indicates the specific heat capacity of air; q represents the ventilation volume of the ventilation system; l Indicates the air leakage rate of the ventilation system; η HR It is the heat recovery rate of the ventilation system;

[0018]

[0019] In the formula, η HC Indicates the power coefficient of the heat pump system; Indicates the power of the heat pump;

[0020]

[0021] In the formula, Indicates the utilization rate of electrical appliances; Appl per Indicates the power of electrical equipment per square meter; A room Indicates the total area of ​​the house; ε appl Indicates the heating power of the electrical appliance;

[0022]

[0023] In the formula, Indicates the occupancy rate of an indoor space at a given time; Body per The heat output of the human body is represented by the body's metabolic rate; Num represents the total number of people in the building.

[0024] Preferably, the building is used as a virtual energy storage node, and considering all factors affecting indoor temperature changes, the thermodynamic model of the building HVAC system is established as follows:

[0025]

[0026] In the formula, ρ represents air density; C ρ V represents the specific heat capacity of air. room Indicates the volume of the house; This represents the room temperature at the current time t;

[0027] Q wall Indicates the heat transfer of the wall; Q win Indicates the heat transfer of the window; Q sol Q represents the heat transfer caused by solar radiation. ven Indicates the heat transfer of the ventilation system; Q hc Indicates the heat transfer capacity of a heat pump; Q app Q represents the radiative heat transfer of electrical equipment. peo Indicates the heat transfer through radiation from the human body;

[0028] Δt represents the room temperature at the next moment t; Δt represents the time interval.

[0029] Preferably, the correction strategy corrects the room temperature at the next moment based on a thermodynamic model, and the controlled time period is divided into time intervals to obtain the virtual energy storage model of the building HVAC system as shown below:

[0030]

[0031] In the formula, N represents the number of time intervals divided within the controlled time period, i is the sequence number of the divided time intervals, and Δt′ is the time interval after division.

[0032] Preferably, considering dynamic electricity prices and weather information, the control problem of the virtual energy storage model is transformed into an MDP model, and the state information of the MDP model is as follows:

[0033]

[0034] in, This represents the room temperature at the current time t;

[0035] Indicates the outdoor temperature at times t, t+1, and t+6;

[0036] This represents the solar radiation intensity at window p at times t, t+1, and t+6.

[0037] T represents the controlled time period;

[0038] Price t Price t+1 This represents the electricity price at times t and t+1.

[0039] Preferably, the control actions of the MDP model are as follows:

[0040]

[0041] in, Indicates the power of the heat pump; This indicates the ventilation volume of the ventilation system.

[0042] Preferably, the optimization objective of the MDP model is:

[0043]

[0044] Where min represents minimization; Abs() is the absolute value function;

[0045] w1 and w2 represent the degree of emphasis on electricity costs and thermal comfort costs, respectively;

[0046] Δt represents the time interval; σ is the thermal comfort cost coefficient; These represent the maximum and minimum values ​​of room temperature.

[0047] Preferably, the reward function of the MDP model is:

[0048]

[0049] Among them, R t Let be the reward function at the current time t.

[0050] Preferably, the PPO algorithm uses an Actor-Critic network structure to optimize and control the MDP model.

[0051] A second aspect of this invention proposes a virtual energy storage modeling and optimization control system for building heating, ventilation, and air conditioning systems, comprising:

[0052] The model building module is used to treat the building as a virtual energy storage node, consider all factors affecting indoor temperature changes, establish a thermodynamic model of the building's HVAC system, and establish a virtual energy storage model of the building's HVAC system based on the thermodynamic model and the correction strategy for the room temperature at the next moment.

[0053] An optimization control module is used to consider dynamic electricity prices and weather information, transforming the control problem of the virtual energy storage model into an MDP model, and using the PPO algorithm to optimize the control of the MDP model.

[0054] A third aspect of the present invention provides a terminal, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to perform operations according to the instructions to execute the steps of the method.

[0055] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method.

[0056] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0057] This invention, based on the spatiotemporal dynamic changes in building energy consumption and load, treats the building as a VES node and comprehensively considers weather factors, human thermal comfort, and dynamic electricity prices. Through the construction of a thermodynamic model and combined with a correction strategy for the room temperature at the next moment, the control of the building's HVAC system is constructed as a virtual energy storage model. The control problem of the virtual energy storage model is transformed into an MDP model, and the PPO algorithm is used to optimize the control of the MDP model. The proposed correction strategy, the virtual energy storage model of the building's HVAC system, and the MDP model can more scientifically reflect changes in indoor temperature. Moreover, during simulation, it significantly reduces the fluctuation of indoor temperature, which can reduce operating costs while ensuring indoor thermal comfort and scheduling and balancing the grid load.

[0058] This invention utilizes deep reinforcement learning technology to control the virtual energy storage model. Compared with other control methods such as rule-based control and model predictive control, deep reinforcement learning technology has the advantages of strong robustness and wide applicability. It can efficiently and flexibly optimize and adjust building loads in multiple scenarios, reduce operating costs and flexibly schedule and balance grid loads while ensuring indoor thermal comfort, providing strong support for the safe and stable operation of the power grid, and realizing precise control and efficient management of building VES resources. Attached Figure Description

[0059] Figure 1 It constitutes the interactive environment of intelligent buildings;

[0060] Figure 2 This is a deep reinforcement learning algorithm framework based on the PPO algorithm;

[0061] Figure 3 Markov decision process for HVAC control systems;

[0062] Figure 4 It is an Actor network structure;

[0063] Figure 5 This is a flowchart of the method of the present invention. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The embodiments described in this application are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of this invention.

[0065] Embodiment 1 of this invention proposes a virtual energy storage modeling and optimization control method for building HVAC systems. It focuses on the dynamic thermal characteristics of buildings, considering the multiple components of the building HVAC system thermodynamic model, establishing mathematical models for each component to form a complete building HVAC system thermodynamic mathematical model; based on the building HVAC system thermodynamic model, a dynamic building virtual energy storage model is formed; based on the above virtual energy storage model, considering weather information, occupant information, and other relevant information, the PPO algorithm based on the MDP process is used to optimize and control the building HVAC system VES model. This can effectively improve the thermal comfort of residents and reduce building operating costs, fully utilize the control potential of the building HVAC system, and achieve optimized integration and efficient utilization of VES resources. Figure 5 As shown, the method specifically includes:

[0066] Step 1: Treat the building as a virtual energy storage node, consider all factors affecting indoor temperature changes, establish a thermodynamic model of the building HVAC system, and establish a virtual energy storage model of the building HVAC system based on the thermodynamic model and the correction strategy for the room temperature at the next moment.

[0067] More preferably, the rise of Virtual Energy Storage (VES) technology provides a new approach to the integration and management of distributed resources. By aggregating distributed energy storage resources to form a virtual energy storage system, flexible scheduling and balancing of grid load can be achieved. Furthermore, building HVAC systems, as a new type of communication infrastructure, possess the potential to serve as virtual energy storage resources.

[0068] This invention specifically analyzes the dynamic changes of building HVAC systems. Based on the thermodynamic changes in buildings, a thermodynamic model of the building HVAC system is established. Based on this, a virtual energy storage model of the building HVAC system is established. The specific thermodynamic model and virtual energy storage modeling of the building HVAC system are as follows:

[0069] 1. Thermodynamic Model of Building HVAC System

[0070] Typical building HVAC systems such as Figure 1 As shown in the figure, there are seven factors affecting indoor temperature changes: wall heat transfer, window heat transfer, solar radiation, ventilation system heat transfer, heat pump, electrical appliance radiation, and human body heat. Modeling is required for these seven influencing factors.

[0071] For heat transfer through walls, the formula is as shown in equation (1). In this formula, N j This refers to the total number of exterior walls; typically, the roof can also be considered an exterior wall. wall This refers to the equivalent heat transfer coefficient of the wall, and different values ​​need to be selected for different wall materials. A wall,j This refers to the area of ​​each wall. and These refer to the temperatures inside and outside the wall, respectively; temperature differences lead to heat transfer. Q wall This refers to the sum of the heat transfer power of all walls.

[0072]

[0073] For heat transfer through windows, the formula is as shown in (2). In this formula, N p This refers to the total number of windows. win This refers to the equivalent heat transfer coefficient of a window. A win,p This refers to the area of ​​each window. Heat transfer through windows is also driven by the temperature difference between the indoor and outdoor areas, Q. win This refers to the total heat transfer from all windows.

[0074]

[0075] For solar radiation, the formula is as shown in (3). Since solar radiation is projected into the room through windows, the solar radiation intensity of each window needs to be considered in the calculation. In this formula, βwin,p This refers to the shading coefficient of each window, which is used to calculate the transmittance of solar radiation and the effect of different window angles. This refers to the intensity of solar radiation at each window. Q sol This refers to the sum of the power of solar radiation projected through all windows.

[0076]

[0077] For heat transfer in a ventilation system, the formula is as shown in (4). In this formula, ρ refers to the air density, C ρ These refer to the specific heat capacity of air; these two parameters describe the physical properties of air. This refers to the ventilation volume of the ventilation system. l The air leakage rate of a ventilation system, η HR It is the heat recovery rate of the ventilation system.

[0078]

[0079] For heat pump heat transfer, the formula is as shown in (5). In this formula, η HC This refers to the power coefficient of a heat pump system. This refers to the power of the heat pump, which can be positive or negative. A positive value indicates that the heat pump system transfers heat to the room, while a negative value indicates that the heat pump system cools the room.

[0080]

[0081] For radiative heat transfer in electrical equipment, the formula is as shown in (6). In this formula, This refers to the utilization rate of electrical appliances, that is, the proportion of all electrical appliances in use at a certain moment. Appl per This refers to the power of electrical equipment per square meter. A room This refers to the total area of ​​the house. ε appl This refers to the heating power of an electrical appliance. The heat transferred by the appliance into the room is denoted by Q. app To express.

[0082]

[0083] For radiative heat transfer in the human body, the formula is as shown in (7). In this formula, This refers to the occupancy rate of an indoor space at a given moment; this number changes over time. per The heat output of the human body is determined by its metabolic rate. Num refers to the total number of people inside the building. Q peo It refers to the sum of the heating power of all heat sources.

[0084]

[0085] Based on the above seven factors affecting indoor temperature changes, we can summarize and generalize the dynamic changes in indoor temperature, which is the thermodynamic model of the building HVAC system, as shown in the following formula:

[0086]

[0087] In the formula: Q wall Indicates the heat transfer of the wall; Q win Indicates the heat transfer of the window; Q sol Q represents the heat transfer caused by solar radiation. ven Indicates the heat transfer of the ventilation system; Q hc Indicates the heat transfer capacity of a heat pump; Q app Indicates the heat transfer of electrical equipment; Q peo Indicates heat transfer between personnel;

[0088] In this formula, V room This refers to the volume of the house. This refers to the current temperature of the room. The formula uses a differential method to calculate the temperature, that is...

[0089] When calculating the temperature at the next moment, formula (9) is used to transform the differential into an integral form, as shown below:

[0090]

[0091] The thermodynamic model of a building HVAC system directly affects the effectiveness of the control method. Therefore, this invention models all the influencing factors of a building HVAC system and establishes a complete indoor temperature change model based on this model. This model is then used to establish a virtual energy storage model for the building HVAC system for optimized control.

[0092] 2. Considering the dynamic changes of the building's HVAC system, the formula for calculating the room temperature at the next moment is revised, and a virtual energy storage model of the building's HVAC system is established:

[0093] The formula for the temperature of a building's HVAC system at the next moment is shown below:

[0094]

[0095] In this formula, the choice of Δt is crucial for calculating a reasonable and accurate indoor temperature. Studies using the above model neglect the characteristic that indoor temperature changes are relatively small over short periods.

[0096] In calculations, Δt is often set to 1 hour. Although this meets the control time period requirement, in practice, the temperature simulated in this way often has extremely strong fluctuations and cannot meet the need to accurately simulate changes in ambient temperature.

[0097] To address the issues of indoor temperature fluctuations and drastic changes, this invention modifies the formula for calculating room temperature at the next moment. The modification strategy involves dividing the controlled time period into time intervals, resulting in the modified formula for the virtual energy storage model of the building HVAC system, as shown in (11):

[0098]

[0099] In formula (11), N refers to the number of time intervals used for calculation within the control period.

[0100] For example, when the control time period is 1 hour, the calculation time interval used in this invention is Δt = 5 minutes, and at this time, N = 12.

[0101] The heat transfer of the surface wall at the current moment; The heat transfer of the window at the current moment; Surface solar radiation heat transfer; The heat transfer of the surface ventilation system at the current moment; The heat transfer rate of the surface heat pump system at the current moment; The heat transfer of the personnel at the current moment; The heat transfer of the meter-type electrical equipment at the current moment.

[0102] Compared to the original formula, the revised formula can more scientifically reflect changes in indoor temperature and significantly reduce the fluctuation of indoor temperature during simulation. Based on the above method, the building HVAC system can be transformed into a virtual energy storage model.

[0103] Step 2: Considering dynamic electricity prices and weather information, the control problem of the virtual energy storage model is transformed into an MDP model, and the PPO algorithm is used to optimize the control of the MDP model.

[0104] More preferably, the operating load of a building's HVAC system is related to indoor temperature and thermal comfort, which affects the overall energy consumption of the building. By accurately modeling the building's HVAC system and flexibly controlling its power consumption, a virtual energy storage model of the building's HVAC system can be established and its adjustable potential can be explored, providing additional energy support to the power grid and participating in the grid's load balancing. However, traditional control methods have significant shortcomings in computational speed and control performance, which directly affects the utilization of the virtual energy storage of the building's HVAC system. Deep reinforcement learning technology has strong flexibility and excellent control performance, and can adapt to various systems. This invention, based on the virtual energy storage model of the building's HVAC system, considers dynamic electricity prices and weather information, and uses a deep reinforcement learning control method to achieve optimal control. Specifically, the established virtual energy storage model of the building's HVAC system is transformed into a Markov decision process (MDP), and the deep reinforcement learning algorithm Proximal Policy Optimization (PPO) is used to achieve optimal control.

[0105] 1. Control problem transformed into MDP

[0106] For intelligent control of HVAC systems, the actual control process can be viewed as a Markov decision process. The control system itself can be considered as an intelligent agent, thus forming a Markov decision process.

[0107] 2. PPO algorithm control based on MDP process

[0108] This invention takes into account both current and future information, and uses the PPO algorithm to optimize and control the established MDP process. Its basic flow is as follows: Figure 2 As shown.

[0109] The specific steps are as follows:

[0110] 1) Establish an MDP model for a building HVAC virtual energy storage control system, such as... Figure 3 As shown. Its state information includes the current state information, such as the solar radiation intensity, outdoor temperature, time, dynamic electricity price, etc., and also includes the state information for future times, as shown in equation (12):

[0111]

[0112] The control action is a combination of the ventilation volume of the ventilation system and the power of the heat pump, which is a continuous interval control as shown in equation (13):

[0113]

[0114] The optimization objective of this control system is to minimize the electricity price while ensuring thermal comfort. Therefore, its formula is shown in (14):

[0115]

[0116] In this formula, min represents the minimization of system cost, indicating that the system can minimize both electricity cost and user thermal comfort cost; w1 and w2 represent the degree of emphasis on electricity cost and thermal comfort cost, respectively. For heat pump power The absolute value of the heat pump power indicates indoor heating when positive and cooling when negative; T refers to the control time period, and price... t This refers to real-time electricity price information, and σ is the thermal comfort cost coefficient. The formula contains two parameters, w1 and w2, which control the degree of emphasis on these two types of costs. Based on this optimization objective, the reward function of MDP is defined as follows:

[0117]

[0118] This transforms the building HVAC virtual energy storage model control into an MDP model.

[0119] 2): The PPO algorithm is used to optimize and control the above MDP model, employing an Actor-Critic network structure, the structure of which is as follows: Figure 4 As shown.

[0120] The PPO algorithm uses an Actor-Critic network structure to optimize and control the MDP model, thereby minimizing the total cost, i.e., the cost of electricity and thermal comfort. According to formula (14), the lower the total cost, the better the algorithm performance.

[0121] This invention evaluates two criteria: electricity cost and thermal comfort cost.

[0122] The cost of thermal comfort is determined using the PMV index, with the average air temperature t at the minimum and maximum PMV values. a As mentioned above

[0123] The general formula for calculating the PMV metric is as follows:

[0124]

[0125] Where M refers to the human metabolic rate, and its unit is W / m³. 2 The human metabolic rate varies depending on the situation, and is generally calculated using the average human metabolic rate. W is the power of the human body's work, and its unit is W / m². 2 Different movements have different power outputs, and the power output within a building is generally negligible. a t is the partial pressure of water vapor, and its unit is Pa.a It is the average air temperature, and its unit is °C. cl This is the clothing coefficient; the more clothes you wear, the higher this value becomes. This is the mean radiant temperature, measured in °C. In buildings, this value is generally taken as the same as the average air temperature, i.e.,

[0126] The total cost includes electricity costs and thermal comfort costs; the lower the total cost, the better the algorithm performs.

[0127] The specific simulation implementation is as follows:

[0128] 1. Parameter settings

[0129] This invention assumes the test house has dimensions of 10 meters (length), 10 meters (width), and 3 meters (height). It sets the house to house a family of three, with the parents working from 8 AM to 6 PM and the child always at home. This means that during the parents' working hours, there is one person in the house, and at other times, there are three. Regarding appliance usage, considering electricity consumption, the usage rate is 0.2% at night and 0.8% during the day. Other parameter settings in the model are shown in Table 1 below.

[0130] Table 1. Parameters used in modeling

[0131]

[0132]

[0133] As shown in the table above, the window-to-wall ratio ranges from 0.2 to 0.4; the value used in this paper is 0.2. Solar radiation refractive index is a key factor affecting indoor temperature. Taking into full account the influence of window materials and other factors, this paper selects 0.13 as the value for simulation. The roof of the building is treated as an exterior wall. Regarding the power of other indoor electrical equipment, considering the power of indoor lighting and other appliances, the total is 23 W / m². 2 This means that when the appliance usage rate is 1, 100m 2 The total power of electrical appliances in the house is 2300W.

[0134] 2. Analysis of the Calculation Results

[0135] The optimization control effect of the deep reinforcement learning method was evaluated using two typical weather conditions: winter and summer. The comparison results of each parameter are shown below:

[0136] Table 2 Summer Experiment Results

[0137] method Total cost Average cost Average temperature Exceeding PMV ratio Standard deviation Average PMV DQN_Future 11471.44 370.05 24.12 67.87％ 4.94 1.17 PPO_noFuture 875.80 28.25 24.38 19.89％ 1.23 0.38 PPO_Future 773.32 24.95 24.56 15.19％ 1.21 0.34

[0138] Table 3. Results of the winter experiment

[0139] method Total cost Average cost Average temperature Exceeding PMV ratio Standard deviation Average PMV DQN_Future 9469.25 338.19 26.07 68.45％ 3.03 0.84 PPO_noFuture 4691.62 167.56 24.86 31.99％ 1.85 0.45 PPO_Future 3752.08 134.60 24.45 24.59％ 1.49 0.36

[0140] From the above results, we can conclude that:

[0141] 1) The method proposed in this invention achieves optimal performance across all parameters. Summer experimental results are shown in Table 2. In terms of total cost and average cost, the proposed method significantly outperforms the Discrete-Motion Control (DQN) method and also outperforms the PPO method, which does not utilize future state information. The improvement is 11.68%. Regarding average temperature, the average temperature obtained by the proposed method is closer to the temperature when PMV is 0, meaning that the temperature at this point is more comfortable. The proportion exceeding PMV indicates the percentage of temperatures exceeding the comfort range specified by PMV. Similarly, the proportion exceeding the specified temperature range by the proposed method is only 15.19%, far lower than the other two frameworks. Compared to the PPO method, which does not contain future state information, the improvement is 23.63%. Standard deviation refers to the standard deviation of temperature, which indicates the stability of the control. The proposed method also achieves the lowest standard deviation in temperature control.

[0142] 2) Table 3 shows the control performance of the proposed method in winter. Similar to summer, the proposed method outperforms the other two methods in winter. In terms of cost savings, the improvement is 24.29%. In terms of comfort, the improvement is 30.09%.

[0143] Therefore, deep reinforcement learning algorithms can achieve efficient and flexible control when dealing with different types of building HVAC systems. The method of this invention can accurately model different types of building HVAC systems and achieve flexible and efficient control.

[0144] Embodiment 2 of the present invention provides a virtual energy storage modeling and optimization control system for a building heating, ventilation and air conditioning system, comprising:

[0145] The model building module is used to treat the building as a virtual energy storage node, consider all factors affecting indoor temperature changes, establish a thermodynamic model of the building's HVAC system, and establish a virtual energy storage model of the building's HVAC system based on the thermodynamic model and the correction strategy for the room temperature at the next moment.

[0146] An optimization control module is used to consider dynamic electricity prices and weather information, transforming the control problem of the virtual energy storage model into an MDP model, and using the PPO algorithm to optimize the control of the MDP model.

[0147] Embodiment 3 of the present invention provides a terminal, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to perform operations according to the instructions to execute the steps of the method.

[0148] Embodiment 4 of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method.

[0149] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0150] This invention, based on the spatiotemporal dynamic changes in building energy consumption and load, treats the building as a VES node and comprehensively considers weather factors, human thermal comfort, and dynamic electricity prices. Through the construction of a thermodynamic model and combined with a correction strategy for the room temperature at the next moment, the control of the building's HVAC system is constructed as a virtual energy storage model. The control problem of the virtual energy storage model is transformed into an MDP model, and the PPO algorithm is used to optimize the control of the MDP model. The proposed correction strategy, the virtual energy storage model of the building's HVAC system, and the MDP model can more scientifically reflect changes in indoor temperature. Moreover, during simulation, it significantly reduces the fluctuation of indoor temperature, which can reduce operating costs while ensuring indoor thermal comfort and scheduling and balancing the grid load.

[0151] This invention utilizes deep reinforcement learning technology to control the virtual energy storage model. Compared with other control methods such as rule-based control and model predictive control, deep reinforcement learning technology has the advantages of strong robustness and wide applicability. It can efficiently and flexibly optimize and adjust building loads in multiple scenarios, reduce operating costs and flexibly schedule and balance grid loads while ensuring indoor thermal comfort, providing strong support for the safe and stable operation of the power grid, and realizing precise control and efficient management of building VES resources.

[0152] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0153] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0154] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0155] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A method for virtual energy storage modeling and optimized control of a building heating, ventilation, and air conditioning system, characterized in that, include: By treating the building as a virtual energy storage node and considering all factors influencing indoor temperature changes, a thermodynamic model of the building's HVAC system is established. Based on this thermodynamic model and a correction strategy for the next moment's room temperature, a virtual energy storage model of the building's HVAC system is then developed. These factors include the heat transfer Q from the walls. wall Window heat transfer Q win Solar radiation heat transfer Q sol Heat transfer Q of ventilation system ven Heat pump heat transfer Q hc Radiated heat transfer Q of electrical equipment app and the heat transfer of the human body by radiation Q peo The details are as follows: In the formula, N j Indicates the total number of exterior walls; U wall A represents the equivalent heat transfer coefficient of the wall; wall,j This indicates the area of ​​each wall. and Let these represent the outdoor temperature and room temperature at the current time t, respectively. In the formula, N p Indicates the total number of windows; U win Indicates the equivalent heat transfer coefficient of the window; A win,p This represents the area of ​​window p. In the formula, β win,p This represents the shading coefficient of window p; This represents the intensity of solar radiation at window p; In the formula, ρ represents air density; C ρ This indicates the specific heat capacity of air; q represents the ventilation volume of the ventilation system. l Indicates the air leakage rate of the ventilation system; η HR It is the heat recovery rate of the ventilation system; In the formula, η HC Indicates the power coefficient of the heat pump system; Indicates the power of the heat pump; In the formula, Indicates the utilization rate of electrical appliances; Appl per Indicates the power of electrical equipment per square meter; A room Indicates the total area of ​​the house; ε appl Indicates the heating power of the electrical appliance; In the formula, Indicates the occupancy rate of an indoor space at a given time; Body per The heat output of the human body is represented by its metabolic rate; Num represents the total number of people in the building. Considering dynamic electricity prices and weather information, the control problem of the virtual energy storage model is transformed into an MDP model, and the PPO algorithm is used to optimize the control of the MDP model. Considering dynamic electricity prices and weather information, the control problem of the virtual energy storage model is transformed into an MDP model, the state information of which is: in, This represents the room temperature at the current time t; Indicates the outdoor temperature at times t, t+1, and t+6; The values ​​represent the solar radiation intensity at window p at times t, t+1, and t+6; T represents the control time period; Price t Price t+1 Indicates the electricity price at times t and t+1; The control actions of the MDP model are as follows: in, Indicates the power of the heat pump; This indicates the ventilation volume of the ventilation system; The optimization objective of the MDP model is: Where min represents minimization; Abs() is the absolute value function; w1 and w2 represent the degree of emphasis on electricity cost and thermal comfort cost, respectively; Δt represents the time interval; and ρ is the thermal comfort cost coefficient. These represent the maximum and minimum values ​​of room temperature; The reward function of the MDP model is: Among them, R t Let be the reward function at the current time t.

2. The virtual energy storage modeling and optimization control method for a building HVAC system according to claim 1, characterized in that: Treating the building as a virtual energy storage node and considering all factors affecting indoor temperature changes, the thermodynamic model of the building's HVAC system is as follows: In the formula, ρ represents air density; C ρ V represents the specific heat capacity of air. room T represents the volume of a house; t in This represents the room temperature at the current time t; Q wall Indicates the heat transfer of the wall; Q win Indicates the heat transfer of the window; Q sol Q represents the heat transfer caused by solar radiation. ven Indicates the heat transfer of the ventilation system; Q hc Indicates the heat transfer capacity of a heat pump; Q app Q represents the radiative heat transfer of electrical equipment. peo Indicates the heat transfer through radiation from the human body; Δt represents the room temperature at the next moment t; Δt represents the time interval.

3. The virtual energy storage modeling and optimization control method for a building HVAC system according to claim 2, characterized in that: The correction strategy is based on a thermodynamic model to correct the room temperature at the next moment. The controlled time period is divided into time intervals, resulting in the virtual energy storage model of the building HVAC system as shown below: In the formula, N represents the number of time intervals divided within the controlled time period, i is the sequence number of the divided time intervals, and Δt′ is the time interval after division.

4. The virtual energy storage modeling and optimization control method for a building HVAC system according to claim 1, characterized in that: The PPO algorithm uses an Actor-Critic network structure to optimize and control the MDP model.

5. A virtual energy storage modeling and optimization control system for a building heating, ventilation, and air conditioning system, comprising the method described in any one of claims 1-4, characterized in that, The control system includes: The model building module is used to treat the building as a virtual energy storage node, consider all factors affecting indoor temperature changes, establish a thermodynamic model of the building's HVAC system, and establish a virtual energy storage model of the building's HVAC system based on the thermodynamic model and the correction strategy for the room temperature at the next moment. An optimization control module is used to consider dynamic electricity prices and weather information, transforming the control problem of the virtual energy storage model into an MDP model, and using the PPO algorithm to optimize the control of the MDP model.

6. A terminal, comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps of the method according to any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method according to any one of claims 1-4.

Images