Building carbon emission intelligent control method based on deep reinforcement learning

Abstract

The invention relates to the technical field of industrial control, in particular to a building carbon emission intelligent control method based on deep reinforcement learning, and the method comprises the steps: collecting environment state data, power grid carbon emission factors and air conditioner real-time power of a building at the current moment; obtaining an environment comprehensive deviation index, wherein the environment comprehensive deviation index represents the difference degree between the environment state data and the ideal environment state data; a carbon emission optimization weight factor is obtained, and the carbon emission optimization weight factor is in inverse correlation with the environment comprehensive deviation index and is in positive correlation with the maximum value of the preset carbon emission optimization weight factor; and a control strategy of the air conditioning system is obtained through the improved deep reinforcement learning model, and operation of the air conditioning system is controlled based on the control strategy. According to the invention, the problem that the traditional building control method still has obvious deficiencies in the multi-target coordination problems of satisfying indoor comfort, optimizing energy utilization and reducing carbon emission is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial control. More particularly, the present application relates to a building carbon emission intelligent control method based on deep reinforcement learning. BACKGROUND

[0002] With the continuous growth of global energy consumption, the continuous expansion of urban building scale, and the increasingly stringent environmental protection regulations and carbon emission constraints, building energy consumption and its related carbon emissions have become one of the key problems restricting sustainable development and the promotion of green buildings. In modern buildings, air conditioning systems, ventilation systems, lighting systems and other electrical equipment are the main sources of energy consumption, among which air conditioning systems have significantly increased the proportion of energy consumption and carbon emissions in the total amount of buildings due to their large operating power, frequent adjustment and close relationship with indoor comfort.

[0003] Traditional building energy consumption management mainly relies on experience-based or rule-based control strategies, such as setting fixed temperature upper and lower limits, timed on-off control or simple linear PID adjustment. Such methods cannot fully consider the complex dynamic relationship between indoor environmental conditions, building load changes, equipment operating characteristics and power grid carbon emission characteristics in actual operation. Although some advanced building automation systems can realize real-time monitoring of environmental parameters such as temperature, humidity and air quality, and remote control of air conditioning units, fans and chiller units, their control logic is mostly static rules or linear adjustment algorithms, lacking unified optimization consideration of indoor environmental comfort, energy efficiency and carbon emission constraints, making it difficult to achieve multi-objective collaborative optimization under complex working conditions.

[0004] In addition, existing control methods also have various technical problems in practical application: on the one hand, due to frequent changes in environmental parameters and equipment states, static or experience-based rules are difficult to respond in time, which can easily lead to indoor temperature being too high or too low, air quality not meeting standards and other comfort problems; on the other hand, fixed control strategies cannot dynamically adjust air conditioning system power and fan operating frequency, resulting in low energy utilization efficiency, serious power waste, and difficulty in achieving effective reduction of carbon emissions, thus leading to obvious deficiencies in traditional building control methods in meeting indoor comfort, optimizing energy utilization and reducing carbon emissions. SUMMARY

[0005] To solve the problem of obvious deficiencies in traditional building control methods in meeting indoor comfort, optimizing energy utilization and reducing carbon emissions in the multi-objective coordination problem, the present application provides the following solutions.

[0006] The application provides a building carbon emission intelligent control method based on deep reinforcement learning, comprising: collecting environment state data of the building at the current time, power grid carbon emission factors and air conditioner real-time power; obtaining an environment comprehensive deviation index, which represents the difference between the environment state data and ideal environment state data; obtaining a carbon emission optimization weight factor, which is inversely related to the environment comprehensive deviation index and positively related to the maximum value of a preset carbon emission optimization weight factor; obtaining a control strategy of the air conditioning system by using an improved deep reinforcement learning model, and controlling the operation of the air conditioning system based on the control strategy; wherein the improved deep reinforcement learning model comprises a reward function, which is inversely related to the power grid carbon emission factor and the air conditioner power and positively related to the carbon emission optimization weight factor.

[0007] The above technical solution combines the environment state, air conditioner energy consumption and power grid carbon emission characteristics organically, constructs a building carbon emission intelligent control method based on deep reinforcement learning, realizes multi-objective optimization of the air conditioning system, guides the deep reinforcement learning model to adaptively adjust the control strategy while optimizing energy consumption and reducing carbon emission, and outputs specific control actions including the outlet water temperature of the water chiller and the operation frequency of the air conditioner terminal fan, so as to realize efficient connection of the strategy and equipment execution, significantly improve the comprehensive performance of the building air conditioning system in energy efficiency management, carbon emission reduction and indoor comfort guarantee, and enhance the adaptability, stability and long-term optimization capability of the system under complex working conditions.

[0008] Further, the environment state data comprises indoor temperature and carbon dioxide concentration.

[0009] Further, the environment state data is standardized.

[0010] Further, the environment comprehensive deviation index at the current time is: , is the temperature at the current time, is a preset ideal temperature, is a preset fluctuation temperature, is a natural constant is a logarithmic function with the natural constant as the base, is the carbon dioxide concentration at the current time, is a preset ideal carbon dioxide concentration, is a weight factor.

[0011] The technical scheme is characterized in that: the indoor temperature deviation and the carbon dioxide concentration deviation are quantified uniformly by constructing an environment comprehensive deviation index, so that the indoor environment comfort and the air quality can be continuously reflected; through normalization and nonlinear mapping processing, the actual influence of the temperature deviation can be highlighted, and the excessive amplification of the carbon dioxide abnormal value can be inhibited; meanwhile, a weight factor is introduced to balance the importance of the two, so that the index can accurately and stably reflect the environmental deviation, and provide reliable state input for the subsequent air conditioning control strategy.

[0012] Further, carbon emission optimization weight factor at the moment is: , is a preset maximum value of the carbon emission weight, is an exponential function with a natural constant as the base, is an adjustment factor, is environment comprehensive deviation index at the moment, is an environment comprehensive deviation index threshold.

[0013] The technical scheme is characterized in that: the carbon emission optimization weight factor is constructed to adaptively change with the environment comprehensive deviation degree, so that the carbon emission constraint can be dynamically adjusted according to the change of the indoor environment state; when the environmental deviation is small, the weight remains at a high level, the low-carbon operation goal is strengthened, and the control strategy is guided to preferentially reduce energy consumption and carbon emission; when the environmental deviation increases and approaches the threshold, the weight is smoothly attenuated, so that the control strategy can preferentially adjust the indoor environment to ensure the comfort and air quality, thereby realizing a reasonable balance between the carbon emission reduction goal and the environmental comfort, and avoiding control shock through the continuous change of the function form, thereby improving the stability and adaptability of the system operation.

[0014] Further, the preset maximum value of the carbon emission weight is 1.

[0015] Further, reward function at the moment is: , is carbon emission optimization weight factor at the moment, is power grid carbon emission factor at the moment, is air conditioner power at the moment, is a preset carbon emission rate, is environment comprehensive deviation index at the moment, is a preset hyperparameter.

[0016] The technical solution combines carbon emission constraints and environmental comfort constraints to construct a reward function, realizes multi-objective optimization of the air conditioning system control strategy, wherein the carbon emission part quantifies the contribution of the air conditioning power according to the dynamically adjusted carbon emission optimization weight and the power grid carbon emission intensity, effectively guides the control strategy to reasonably distribute the power load while meeting the low-carbon operation target, and the environmental comfort part punishes the degree of deviation from the ideal state through the comprehensive environmental deviation index, so that the control strategy makes decisions on the premise of guaranteeing indoor temperature and air quality, thereby improving the overall operation performance and optimization efficiency of the air conditioning system in energy management, carbon emission reduction and indoor environmental comfort guarantee.

[0017] Further, the preset hyperparameter is 0.1.

[0018] Further, the DDC controller is used to control the operation of the air conditioning system.

[0019] Further, the improved deep reinforcement learning model is used to obtain the control strategy of the air conditioning system, specifically: a deep deterministic policy gradient (DDPG) algorithm is used to train the Actor network of the agent; the network parameters of the Actor network are updated based on the reward value output by the reward function corresponding to the current time of the air conditioning system; and a control action instruction is output by the updated Actor network, the control action instruction at least including a chilled water unit outlet water temperature set value and an air conditioning terminal fan frequency set value.

[0020] The above technical solution combines the improved deep reinforcement learning model with air conditioning system control, realizes intelligentization and adaptive optimization of the control strategy, wherein the deep deterministic policy gradient algorithm is used to train the policy network, so that the agent can automatically generate the optimal control action according to the current environment state and the comprehensive reward function, considering carbon emission and energy optimization, and also considering indoor environmental comfort, and through continuous updating of the network parameters, the control strategy can adapt to different operating conditions, and specific control instructions including chilled water unit outlet water temperature and air conditioning terminal fan operating frequency are output, thereby realizing seamless connection of the strategy and actual execution, effectively improving the comprehensive performance of the air conditioning system in energy management, carbon emission reduction and indoor comfort guarantee, and improving the response speed, stability and long-term optimization ability of the system.

[0021] The present application has the following advantages: This invention constructs a building carbon emission intelligent control method based on deep reinforcement learning, realizing multi-objective, adaptive optimization control of air conditioning systems. By collecting environmental conditions, grid carbon emission factors, and air conditioning power information in real time, the method quantifies the degree of environmental deviation and dynamically adjusts the carbon emission optimization weights. This enables the control strategy to balance energy consumption and carbon emission targets while ensuring indoor temperature and air quality, thereby significantly improving the overall performance of building air conditioning systems in terms of energy conservation and emission reduction, environmental comfort, and system response stability. It also enhances the system's adaptability, optimization efficiency, and long-term operational reliability. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating an intelligent building carbon emission control method based on deep reinforcement learning according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the comparison of total daily carbon emissions before and after the improvement of the deep reinforcement learning model according to an embodiment of the present invention. Detailed Implementation

[0023] An embodiment of a building carbon emission intelligent control method based on deep reinforcement learning.

[0024] like Figure 1 As shown in the flowchart, an embodiment of the present invention provides a method for intelligent control of building carbon emissions based on deep reinforcement learning, which includes the following steps: S1: Collect environmental status data of the building at the current moment, power grid carbon emission factor, and real-time power of air conditioning.

[0025] In a preferred embodiment, the environmental status data includes at least indoor temperature and carbon dioxide concentration. Indoor air temperature is used to reflect the effect of the air conditioning system on regulating the indoor thermal environment, and carbon dioxide concentration is used to characterize indoor air quality and the impact of human activities on the environment. The aforementioned environmental status data is acquired in real time through sensors or building automation systems deployed in different functional areas of the building, and updated according to a preset sampling period to ensure the timeliness and accuracy of the status information. For the collected environmental status data, a standardized processing method is used to perform a unified scale conversion on different physical quantities to eliminate the differences in the dimensions, value ranges and numerical amplitudes of various state variables, and to avoid adverse effects on the learning and decision-making process of the subsequent control model due to inconsistent numerical scales. Simultaneously, the grid carbon emission factor corresponding to the current moment is obtained. This grid carbon emission factor is used to characterize the carbon emission intensity implied by unit power consumption. It can be directly obtained based on the benchmark carbon emission factor of the target area grid, or it can be obtained by weighted calculation based on the power generation ratio of different power generation types in the grid and their corresponding unit carbon emission coefficients, thereby reflecting the impact of changes in the grid energy structure on carbon emission levels at different times. In addition, real-time power data of the air conditioning system under the current operating state is collected to characterize the instantaneous energy consumption level and load status of the air conditioning system. Finally, the normalized environmental state data, grid carbon emission factor, and real-time power data of the air conditioning are integrated to construct a state vector that can simultaneously reflect comfort requirements, energy consumption levels, and carbon emission constraints, providing a complete, continuous, and physically meaningful state input for the subsequent generation and optimization of air conditioning control strategies based on reinforcement learning.

[0026] S2: Obtain the comprehensive environmental deviation index and carbon emission optimization weighting factor.

[0027] In a preferred embodiment, Environmental Deviation Index at Current Time for: , for The temperature at that moment The preset ideal temperature, For the preset fluctuation temperature, For the natural constant Logarithmic function with base 0. for The carbon dioxide concentration at that time The preset ideal carbon dioxide concentration, This is the weighting factor.

[0028] By constructing an environmental comprehensive deviation index, the indoor environmental status of a building at any given time is uniformly and quantitatively characterized. Temperature deviation and carbon dioxide concentration deviation are simultaneously incorporated into the same evaluation framework. The differences in the dimensions and magnitude of change of different environmental parameters are eliminated through normalization and nonlinear mapping, so that the degree of environmental deviation can be accurately reflected by a single continuous value. Among these measures, relative bandwidth normalization is used for temperature deviations to highlight the actual impact beyond the comfort range. Logarithmic functions are introduced to compress and map carbon dioxide concentration deviations, effectively reducing the excessive amplification of evaluation results by high-concentration outliers, thus improving the robustness and stability of the indicators to environmental changes. By introducing a weighting adjustment mechanism, the importance of thermal comfort and air quality in the comprehensive evaluation can be flexibly balanced according to different application scenarios. This ensures that the obtained comprehensive environmental deviation index can sensitively reflect the declining trend of environmental comfort without being distorted by drastic fluctuations in a single parameter. This provides a physically meaningful, numerically stable, and continuously optimizable environmental state input for subsequent control strategies or intelligent decision-making models.

[0029] Carbon emission optimization weighting factor at any given time for: , To set the maximum value for the preset carbon emission weights, For the natural constant An exponential function with base 0. As a regulating factor, for The overall environmental deviation index at any given time. The maximum value of the preset carbon emission weight is 1, which is the threshold for the comprehensive environmental deviation index. Of course, it can also be set according to the actual situation.

[0030] By introducing a carbon emission optimization weight factor that adaptively changes with the degree of environmental deviation, carbon emission constraints can be dynamically adjusted according to changes in indoor environmental conditions. When the indoor environment deviates slightly from the ideal state, the carbon emission optimization weight remains at a high level, thereby reinforcing the low-carbon operation target and guiding the control strategy to prioritize reducing energy consumption and carbon emissions while ensuring basic comfort. When the degree of environmental deviation gradually increases and approaches or exceeds a set threshold, the weight factor decays smoothly through a nonlinear function, automatically weakening the impact of carbon emission constraints on the decision-making process. This allows the control strategy to prioritize correcting environmental comfort and air quality issues, avoiding significant deterioration of the indoor environment due to overemphasis on low-carbon targets. By using a continuous and smooth function form to implement weight changes, control strategy oscillations or frequent switching caused by sudden weight changes can be effectively avoided, improving the stability and controllability of system operation. At the same time, a reasonable, adjustable, and dynamic balance that meets actual operating needs is formed between carbon emission reduction targets and comfort assurance targets, thereby improving the overall optimization effect and engineering applicability of the air conditioning system under complex operating conditions.

[0031] S3: Obtain the control strategy of the air conditioning system using an improved deep reinforcement learning model, and control the operation of the air conditioning system based on the control strategy.

[0032] In a preferred embodiment, the improved deep reinforcement learning model includes a reward function. Reward function at time for: , for The carbon emission optimization weighting factor at any given time for Current power grid carbon emission factor for Current air conditioner power To preset the carbon emission rate, for The overall environmental deviation index at any given time. The preset hyperparameter is 0.1, but it can also be set according to the actual situation.

[0033] like Figure 2 As shown in the figure, this invention presents a comparison of the cumulative carbon emissions throughout the day before and after the improvement of the deep reinforcement learning model in an embodiment of the invention.

[0034] By organically combining carbon emission constraints and environmental comfort constraints to construct a reward function, multi-objective optimization of the air conditioning system control strategy is achieved. The carbon emission component quantifies the contribution of dynamically adjusted carbon emission optimization weights and grid carbon emission intensity to air conditioning power, effectively guiding the agent to rationally allocate power load while meeting energy efficiency and low-carbon objectives. The environmental comfort component penalizes deviations from the ideal state through a comprehensive environmental deviation index, ensuring the control strategy makes decisions while guaranteeing indoor temperature and air quality. This reward function, employing a continuous and differentiable nonlinear form, smoothly adjusts the weight relationship between carbon emission constraints and comfort constraints, avoiding control oscillations caused by constraint conflicts. It also enhances the stability, adaptability, and optimization efficiency of the deep reinforcement learning model under complex operating conditions, thereby comprehensively improving the air conditioning system's overall technical performance in energy consumption control, carbon emission reduction, and ensuring indoor environmental comfort.

[0035] An improved deep reinforcement learning model is used to obtain the control strategy for the air conditioning system. Specifically, the Actor network of the agent is trained using the Deep Deterministic Policy Gradient (DDPG) algorithm; the network parameters of the Actor network are updated based on the reward value output by the reward function corresponding to the current moment of the air conditioning system; the updated Actor network outputs control action commands, which include at least the chiller outlet water temperature setpoint and the air conditioning terminal fan frequency setpoint. The operation of the air conditioning system is then controlled using a DDC controller.

[0036] By combining an improved deep reinforcement learning model with air conditioning system control, intelligent and multi-objective operation optimization is achieved. A deep deterministic policy gradient algorithm is used to train the agent's policy network, enabling it to automatically generate optimal control actions based on the current environmental state and the comprehensive reward function. This considers both carbon emission optimization and indoor environmental comfort and air conditioning energy efficiency. Continuous updates to network parameters allow the control strategy to adapt to different operating conditions, outputting specific control commands including chiller outlet water temperature settings and air conditioning terminal fan frequency settings. Simultaneously, a direct digital controller (DDC) precisely controls the air conditioning system's actions, achieving seamless integration between intelligent strategy and actual execution. This significantly improves the overall performance of the air conditioning system in terms of energy management, carbon emission reduction, and indoor comfort assurance, enhancing the system's adaptability, response speed, and long-term stability. Furthermore, it provides a quantifiable and sustainable intelligent control solution for multi-objective optimization.

[0037] This invention constructs a building carbon emission intelligent control method based on deep reinforcement learning, achieving multi-objective optimization and intelligent operation of air conditioning systems. The comprehensive environmental deviation index quantifies the difference between the indoor environmental state and the ideal state, enabling the control strategy to adaptively adjust while ensuring indoor temperature and air quality. The dynamic adjustment mechanism of the carbon emission optimization weight factor automatically balances low-carbon operation and comfort requirements based on environmental conditions. The reward function, by comprehensively considering carbon emission levels, air conditioning power, and environmental deviation, guides the deep reinforcement learning model to optimize energy consumption and reduce carbon emissions while also considering environmental comfort. The deep deterministic policy gradient algorithm continuously updates the policy network parameters, enabling the agent to adaptively generate optimal control actions under different operating conditions. It outputs specific control commands, including chiller outlet water temperature and air conditioning terminal fan frequency, which are precisely executed by a direct digital controller. This achieves efficient integration between the strategy and actual equipment operation, effectively improving the comprehensive performance of building air conditioning systems in energy efficiency management, carbon emission reduction, and indoor comfort assurance. It enhances the system's stability, response speed, and long-term optimization capabilities under complex operating conditions, while providing an intelligent, sustainable, and quantifiable control solution for multi-objective coordinated optimization.

[0038] In the description of this specification, "multiple" or "several" means at least two, such as two, three or more, unless otherwise explicitly specified.

[0039] While this specification has shown and described numerous embodiments of the invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in the practice of this invention.

Claims

1. A building carbon emission intelligent control method based on deep reinforcement learning, characterized in that, include: Collect environmental status data of buildings at the current moment, power grid carbon emission factors, and real-time power consumption of air conditioners; Obtain the comprehensive environmental deviation index, which characterizes the degree of difference between environmental state data and ideal environmental state data; Obtain carbon emission optimization weighting factors, which are inversely correlated with the comprehensive environmental deviation index and positively correlated with the maximum value of the preset carbon emission optimization weighting factors; An improved deep reinforcement learning model is used to obtain the control strategy of the air conditioning system, and the operation of the air conditioning system is controlled based on the control strategy. The improved deep reinforcement learning model includes a reward function that is inversely correlated with the power grid carbon emission factor and air conditioning power, and positively correlated with the carbon emission optimization weight factor.

2. The intelligent building carbon emission control method based on deep reinforcement learning according to claim 1, characterized in that, The environmental status data includes indoor temperature and carbon dioxide concentration.

3. The intelligent building carbon emission control method based on deep reinforcement learning according to claim 1, characterized in that, The environmental status data is then standardized.

4. The intelligent building carbon emission control method based on deep reinforcement learning according to claim 1, characterized in that, Environmental Comprehensive Deviation Index at Current Time for: , for The temperature at that moment The preset ideal temperature, For the preset fluctuation temperature, For the natural constant Logarithmic function with base 0. for The carbon dioxide concentration at that time The preset ideal carbon dioxide concentration, This is the weighting factor.

5. The intelligent building carbon emission control method based on deep reinforcement learning according to claim 1, characterized in that, Carbon emission optimization weighting factor at any given time for: , To set the maximum value for the preset carbon emission weights, For the natural constant An exponential function with base 0. As a regulating factor, for The overall environmental deviation index at any given time. This is the threshold for the comprehensive environmental deviation index.

6. The intelligent building carbon emission control method based on deep reinforcement learning according to claim 5, characterized in that, The maximum value of the preset carbon emission weight is 1.

7. The intelligent building carbon emission control method based on deep reinforcement learning according to claim 1, characterized in that, Reward function at time for: , for The carbon emission optimization weighting factor at any given time for Current power grid carbon emission factor for Current air conditioner power To preset the carbon emission rate, for The overall environmental deviation index at any given time. These are the preset hyperparameters.

8. The intelligent building carbon emission control method based on deep reinforcement learning according to claim 7, characterized in that, The preset hyperparameter is 0.

1.

9. The intelligent building carbon emission control method based on deep reinforcement learning according to claim 1, characterized in that, The operation of the air conditioning system is controlled using a DDC controller.

10. The intelligent building carbon emission control method based on deep reinforcement learning according to claim 1, characterized in that, The control strategy for the air conditioning system is obtained using an improved deep reinforcement learning model, specifically: The Actor network of the agent is trained using the Deep Deterministic Policy Gradient (DDPG) algorithm; The network parameters of the Actor network are updated based on the reward value output by the reward function corresponding to the current moment of the air conditioning system. The updated Actor network outputs control action commands, which include at least the chiller unit's outlet water temperature setpoint and the air conditioning terminal fan's frequency setpoint.

Images