Learning-driven automatic building energy management method

Abstract

The invention provides a learning-driven automatic building energy management method, and the method comprises the specific steps: 1, building a combinable virtual energy system digital twin environment according to the structures and characteristics of different energy devices; 2, determining a state space, an action space and a reward function of the system in the virtual environment; and 3, by using a DQN algorithm, through a reinforcement learning mode, optimizing control strategies under different configurations, and reasonably setting hyper-parameters. And 4, verifying the performance of the control strategy optimized based on the DQN algorithm in the virtual energy system through a numerical experiment. According to the invention, the combinable digital twinning virtual simulation environment is constructed, and the deep reinforcement learning algorithm is combined, so that the intelligent regulation and control of the energy system are realized. The research shows that even in the case of system configuration difference and data acquisition difficulty, the method provided by the invention can still generate an efficient control strategy, so that the accurate adjustment of the temperature and the remarkable reduction of the energy consumption are realized.

Description

Technical Field

[0001] The present invention relates to the field of energy management and automation control, and relates to a learning-based automated building energy management technology; specifically, it is a learning-driven automated building energy management method based on combinable virtual digital twins and Deep Q Network (DQN) algorithms. Background Art

[0002] With the continuous growth of global energy demand and the continuous increase of building energy consumption, how to achieve efficient and intelligent building energy management has become the research focus in the current energy field. The energy consumption in buildings is mainly concentrated on systems such as heating, ventilation, air conditioning, and lighting. The operating efficiency of these systems directly affects the energy utilization efficiency and the comfort of the indoor environment. Therefore, proposing an innovative building energy management system can not only effectively reduce energy costs but also significantly reduce the environmental burden.

[0003] Currently, most traditional building energy management technologies are based on automated control systems with fixed rules and preset parameters. These systems can achieve basic energy management functions to a certain extent, but they usually show great limitations when facing the complex dynamic changes of the building environment and diverse energy demands. In addition, the optimization of traditional methods usually relies on a large amount of actual operation data. However, due to the difficulty, long cycle, and limited data volume of data collection, it is difficult for existing control systems to meet the needs of real-time learning and adaptive optimization. Summary of the Invention

[0004] In view of the above problems, the present invention provides a learning-driven automated building energy management method based on combinable virtual digital twins and DQN algorithms.

[0005] The technical solution of the present invention is as follows: A learning-driven automated building energy management method based on combinable virtual digital twins and DQN algorithms of the present invention includes the following steps:

[0006] Step (1), according to the structures and characteristics of different energy devices, establish a combinable virtual energy system digital twin environment to simulate the dynamic processes of temperature changes and energy consumption fluctuations in the real environment;

[0007] Step (2), in this virtual environment, clarify the state space, action space, and reward function of the system to ensure that the simulation process can accurately reflect the operating characteristics of the energy system;

[0008] Step (3), adopt the DQN algorithm to learn the optimal control strategy under different configurations of the system, and set appropriate hyperparameters to enable the intelligent agent to interact and learn in the virtual digital twin environment;

[0009] Step (4): Verify the performance of the control strategy optimized based on the DQN algorithm in the virtual energy system through numerical experiments.

[0010] Further, in step (1), the composable virtual energy system digital twin environment can flexibly adjust the settings of various devices and parameters according to the different configurations of the actual building. This virtual environment dynamically combines devices such as boilers, air conditioning units, radiators, and heating coils to adapt to different building types and actual usage requirements. At the same time, this environment includes a temperature dynamic model of key components, which can accurately simulate the heat exchange and energy consumption changes between various parts.

[0011] Further, in step (2), the state space S consists of multiple state variables, including the heater temperature T H , the boiler temperature T B , the outdoor ambient temperature T O , the indoor ambient temperature T R , the coil temperature T C , and the air handling unit temperature T A , as well as the boiler setpoint temperature T set , that is, S = {T H , T B , T O , T R , T C , T A , T set}; The action space A includes control signals that can be used to adjust the system temperature, specifically including the control signal U B of the boiler and the control signal U C of the coil, that is, A = {U B , U C}; The reward function r is used to measure the performance of the control strategy and depends on energy consumption and temperature tracking error, r = -(α·E + β·ΔT). Among them, E represents energy consumption, ΔT represents temperature tracking error (i.e., the deviation between the actual temperature and the setpoint temperature), and α and β are weight coefficients used to balance the importance of energy consumption and temperature tracking. Both energy consumption E and temperature tracking error ΔT are functions of the control signals U B , U C and the temperature states T H , T B , T O , T R , T C , T A , T set .

[0012] Further, in step (3), the hyperparameters of the DQN algorithm include network structure, learning rate, discount factor, experience replay buffer size, and target network update frequency, etc.

[0013] Further, in step (4), the numerical experiment sets multiple parameter combinations, covering factors such as different outdoor temperature change patterns, initial equipment states, and set temperatures. By recording and analyzing the experimental data, the performance of the control strategy optimized based on the DQN algorithm in accurately tracking the target temperature and the effect of reducing energy consumption under different working conditions are evaluated.

[0014] The beneficial effects of the present invention are as follows:

[0015] 1. By constructing a composable digital twin environment of the virtual energy system, the present invention can simulate the dynamics of different energy devices under actual building configurations, providing rich scenarios for the training of control strategies and solving the problems of difficult data collection and high costs in the actual environment;

[0016] 2. By defining the state space, action space, and reward function, the present invention can meet the dual requirements of energy conservation and temperature control;

[0017] 3. Using the DQN algorithm, the present invention can learn the optimal control strategy in the virtual environment, enhancing the applicability and effectiveness of the control strategy. Description of the Drawings

[0018] Figure 1 is the overall flowchart of the present invention. Detailed Embodiments

[0019] To more clearly illustrate the technical solution of the present invention, the technical solution of the present invention will be further described in detail below with reference to the drawings:

[0020] As Figure 1 described, a learning-based automated building energy management technology in this embodiment, as Figure 1 shown, includes the following steps:

[0021] Step 1: According to the structures and characteristics of different energy devices, a composable digital twin environment of the virtual energy system is established to simulate the dynamic processes of temperature changes and energy consumption fluctuations in the real environment;

[0022] In this embodiment, the composable digital twin environment of the virtual energy system can flexibly adjust the settings of various devices and parameters according to different configurations of the actual building. This virtual environment dynamically combines devices such as boilers, air conditioning units, radiators, and heating coils to adapt to different building types and actual usage requirements. At the same time, this environment includes a temperature dynamic model of key components, which can accurately simulate the heat exchange and energy consumption changes between various parts. The temperature changes of each part follow the following mathematical formulas:

[0023] Boiler temperature change and control:

[0024]

[0025] U B = K B (T set - T B )

[0026] Air handling unit temperature change:

[0027]

[0028] Coil temperature change and control:

[0029]

[0030] U C = K C (T B - T C )

[0031] Wherein, T set is the set point temperature of the boiler, T H , T B , T O , T R , T C , T A are the temperatures of the heater, boiler, outdoor, indoor, coil and air handling unit, and K HB , K B , K OB , K RB , K CB , K CA , K OA , K RA , K AC , K C are the control gain coefficients of each part.

[0032] Step 2: In this virtual environment, clarify the state space, action space and reward function of the system to ensure that the simulation process can accurately reflect the operating characteristics of the energy system;

[0033] In this embodiment, the state space S consists of multiple state variables, including the heater temperature, boiler temperature, outdoor ambient temperature, indoor ambient temperature, coil temperature and air handling unit temperature, as well as the boiler set point temperature, i.e., S = {T H , T B , T O , T R , T C , T A , T set}; The action space A includes control signals that can be used to adjust the system temperature, specifically including the control signal U of the boiler B and the control signal U of the coil C , that is, A = {U B , U C}; The reward function r is used to measure the performance of the control strategy and depends on energy consumption and temperature tracking error, r = -(α·E + β·ΔT). Among them, E represents energy consumption, ΔT represents temperature tracking error (i.e., the deviation between the actual temperature and the set-point temperature), and α and β are weight coefficients used to balance the importance of energy consumption and temperature tracking. Both energy consumption E and temperature tracking error ΔT are functions of the control signals U B , U C and the temperature state T H , T B , T O , T R , T C , T A , T set .

[0034] Step 3: Use the DQN algorithm to learn the optimal control strategy under different configurations of the system, and set appropriate hyperparameters to enable the agent to interact and learn in the virtual digital twin environment;

[0035] In this embodiment, the hyperparameters of the DQN algorithm include network structure, learning rate, discount factor, experience replay buffer size, and target network update frequency, etc.

[0036] The update rule of the DQN algorithm is as follows:

[0037]

[0038] Among them, θ is the network parameter, α is the learning rate, θ - is the target network parameter, γ is the discount factor, s′ and a′ are the next state and action respectively. In this way, the agent learns how to adjust the boiler control signal U B and the coil control signal U C to maximize the reward function and achieve optimal temperature control and energy-saving effects.

[0039] Step 4: Through numerical experiments, verify the performance of the control strategy optimized based on the DQN algorithm in the virtual energy system.

[0040] In this embodiment, numerical experiments are conducted by setting various parameter combinations, covering different outdoor temperature change patterns, initial equipment states, set temperatures, and other factors. By recording and analyzing the experimental data, the performance of the control strategy optimized based on the DQN algorithm in accurately tracking the target temperature and the effect of reducing energy consumption under different working conditions are evaluated. The experimental results are used to evaluate the applicability and effectiveness of the control strategy in meeting the energy-saving and temperature-control requirements in a virtual environment. Through the above steps, the present invention can achieve accurate temperature tracking and significantly reduce energy consumption, thereby meeting the energy-saving and temperature-control requirements in a virtual environment.

[0041] Finally, it should be understood that the embodiments described in the present invention are only used to illustrate the principles of the embodiments of the present invention; other variations may also fall within the scope of the present invention; therefore, by way of example and not limitation, alternative configurations of the embodiments of the present invention may be considered consistent with the teachings of the present invention; accordingly, the embodiments of the present invention are not limited to the embodiments specifically introduced and described in the present invention.

Claims

1. A learning-driven automated building energy management method, characterized in that: The following steps are involved: Step (1), according to the structure and characteristics of different energy devices, establish a combinable virtual energy system digital twin environment to simulate the dynamic process of temperature change and energy consumption fluctuation in the real environment; Step (2), in the virtual energy system digital twin environment, clarify the system's state space, action space and reward function to ensure that the simulation process can accurately reflect the operating characteristics of the energy system; Step (3), using the DQN algorithm to learn the optimal control strategy under different configurations of the system, and setting appropriate hyperparameters to enable the intelligent agent to interactively learn in the virtual digital twin environment; Step (4): Verify the performance of the control strategy optimized based on the DQN algorithm in the virtual energy system through numerical experiments.

2. A learning driven automated building energy management method according to claim 1, characterized in that: In step (1), the combinable virtual energy system digital twin environment flexibly adjusts the parameters of various equipment according to the different configurations of the actual building; the virtual energy system digital twin environment dynamically combines boilers, air-conditioning units, radiators, and heating coil equipment to adapt to different building types and actual usage needs; at the same time, the virtual energy system digital twin environment includes temperature dynamic models of key components, which can accurately simulate heat exchange and energy consumption changes between various parts.

3. The learning-driven automated building energy management method according to claim 1, characterized in that: In step (2), the state space S consists of a plurality of state variables, including the heater temperature T H , boiler temperature T B , Outdoor ambient temperature T O , Indoor ambient temperature T R , coil temperature T C and the air handling unit temperature T A , and the boiler set point temperature T set , that is, S = {T H ,T B ,T O ,T R ,T C ,T A ,T set }; The action space A includes control signals that can be used to adjust the system temperature, specifically including the boiler control signal U B and the coil control signal U C , that is, A={U B ,U C }; The reward function r is used to measure the performance of the control strategy, which depends on the energy consumption and the temperature tracking error, r = -(α·E+β·ΔT); where E represents the energy consumption, ΔT represents the temperature tracking error, that is, the deviation between the actual temperature and the set point temperature, α and β are weight coefficients used to balance the importance of energy consumption and temperature tracking; both the energy consumption E and the temperature tracking error ΔT are the control signal U B , U C and temperature state T H ,T B ,T O ,T R ,T C ,T A ,T set function.

4. The learning-driven automated building energy management method according to claim 1, characterized in that: In step (3), the hyperparameters of the DQN algorithm include network structure, learning rate, discount factor, experience replay buffer size and target network update frequency.

5. The learning-driven automated building energy management method according to claim 1, characterized in that: In step (4), the numerical experiment covers different outdoor temperature change modes, equipment initial states and set temperature factors by setting a variety of parameter combinations; by recording and analyzing the experimental data, the performance of the control strategy optimized based on the DQN algorithm in accurately tracking the target temperature and the effect of reducing energy consumption under different working conditions are evaluated.

Images