Active Data Generation Method for Optimal Intelligent Control Lifecycle of HVAC Systems

By combining expert systems and reinforcement learning, the problem of intelligent control adaptability of air conditioning systems throughout their life cycle was solved, achieving efficient control and performance optimization throughout the entire life cycle, and improving the control accuracy and operating efficiency of the air conditioning system.

CN116608573BActive Publication Date: 2026-11-17SHANGHAI QUNWEI XINKONG INTELLIGENT TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310704353.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-06-14
Publication Date
2026-11-17
Estimated Expiration
2043-06-14

AI Technical Summary

Technical Problem

Existing intelligent control methods for air conditioning systems have limited adaptability and control effectiveness at different stages of the air conditioning system's life cycle, especially during the commissioning, trial operation, and stable operation phases, where they cannot achieve efficient energy management and personalized comfort control throughout the entire life cycle.

Method used

This approach combines expert systems and reinforcement learning to achieve intelligent control of the air conditioning system throughout its entire lifecycle. This is achieved through data generation during the commissioning phase and phased switching during the stable intelligent control phase. During commissioning, the expert system is used to detect sensors and actuators. During trial operation, a high-quality dataset is generated. In the stable intelligent control phase, expert models and reinforcement learning methods are combined to adaptively adjust the control strategy.

Benefits of technology

It achieves high-efficiency control and operation of the air conditioning system throughout its entire life cycle, ensuring continuous optimization of system performance and meeting users' needs for energy efficiency and comfort.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116608573B_ABST
    Figure CN116608573B_ABST
Patent Text Reader

Abstract

This invention discloses a method for proactively generating data within the optimal intelligent control lifecycle of a heating, ventilation, and air conditioning (HVAC) system. The method includes: using an expert system for automatic detection; providing a high-quality dataset for the stable intelligent control phase, with the expert system providing positive feedback to reinforcement learning to assist in generating the high-quality dataset; the reinforcement learning intelligent control system generating a control dataset at the optimal control frequency; and employing a combined model-based and reinforcement learning intelligent control method. During the first cooling and heating season of the HVAC system, reinforcement learning is used for intelligent control, while from the second cooling and heating season onwards, the method switches to model-based control. When the system's operating data for a cooling and heating season is not in the historical database, the system automatically switches to reinforcement learning for control. This invention generates a high-quality dataset during the trial operation phase, adaptively adjusts the control strategy, and improves the system's control performance under different operating conditions.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of HVAC control technology, and in particular relates to a method for actively generating data within the optimal intelligent control lifecycle of HVAC systems. Background Technology

[0002] Existing intelligent control technologies for air conditioning systems mainly include sensor-based feedback control methods, model predictive control (MPC) methods, and intelligent control methods based on a single AI algorithm. Sensor-based feedback control methods collect data from sensors such as temperature and humidity, adjusting the air conditioning system according to set thresholds. However, these solutions are slow to respond to environmental changes, and their control strategies are relatively simple, making it difficult to achieve efficient energy management and personalized comfort control.

[0003] Model predictive control (MPC) is used to establish a mathematical model of the air conditioning system and adjust the control strategy according to predicted environmental changes and user needs. However, the MPC method requires an accurate system model and a controller with high computational power, and its adaptability to nonlinear and time-varying systems is limited.

[0004] Intelligent control methods based on a single AI algorithm utilize AI algorithms such as machine learning and neural networks to intelligently control air conditioning systems. However, while single AI algorithms often perform well under specific scenarios and data distributions, they may have limitations when processing diverse and complex air conditioning system data and require high data quality.

[0005] A single AI algorithm may have limitations when processing complex air conditioning system data, affecting the control effect, and is mainly suitable for the stable operation phase of the air conditioning system and the phase with good data quality.

[0006] At present, the control of HVAC systems is mainly based on PID feedback control, and AI intelligent control is concentrated in the stable operation stage, which cannot be adapted to the commissioning, operation, adjustment and stable operation stages throughout the life cycle of air conditioning.

[0007] The requirements for artificial intelligence control algorithms at different stages of the air conditioning system's operation and control lifecycle were not considered, therefore they can only be applied to single scenarios and single time periods. Summary of the Invention

[0008] In view of this, the present invention proposes a method for proactively generating data within the optimal intelligent control lifecycle of HVAC systems. Based on a full understanding of the operation, data and control characteristics of the air conditioning system throughout its lifecycle, the method controls the operation of the HVAC system to optimize the generation and connection of data at each stage of the lifecycle, thereby achieving optimal intelligent control throughout the entire lifecycle.

[0009] The present invention discloses a method for actively generating data within the optimal intelligent control lifecycle of HVAC systems, comprising the following steps:

[0010] Commissioning phase: Generate basic active data for the trial operation phase, including: testing all sensors and actuators of the air conditioning system; using an expert system for automatic testing to evaluate the data range, controllability and other key parameters of the sensors and actuators;

[0011] Trial operation phase: To provide high-quality datasets for the stable intelligent control phase, the expert system is responsible for providing positive feedback to the reinforcement learning to assist the reinforcement learning control of the HVAC system in generating high-quality datasets; This phase uses the data output from the debugging phase, and the reinforcement learning intelligent control system generates control datasets at the optimal control frequency;

[0012] Stable intelligent control phase: An intelligent control method combining model and reinforcement learning is adopted. During the first cooling and heating season of the HVAC system, reinforcement learning is used for intelligent control. Starting from the second cooling and heating season, the model-based control method is switched to the model-based control method. When the system operation data is not in the historical database during a cooling and heating season, the system automatically switches to the reinforcement learning method for control.

[0013] Furthermore, a smooth transition between each stage is ensured to facilitate the seamless transition of the air conditioning system between different phases; and by providing a stable and reliable data source for later stable intelligent control in the early stages of the life cycle, the effectiveness of intelligent control throughout the entire life cycle is guaranteed.

[0014] Furthermore, K-means clustering is used to determine whether the system's operating data within a cooling and heating season is in the historical database.

[0015] Furthermore, the stable intelligent control stage primarily uses model-based control to improve control effectiveness, supplemented by reinforcement learning for environmental sample generation and basic range control.

[0016] Furthermore, the expert system is a set of operating parameters of the HVAC system mechanism, while the reinforcement learning system controls the outlet water temperature of the chiller unit within a certain range and continuously generates a dataset within that range.

[0017] Furthermore, during the debugging phase, the system is adjusted and optimized as necessary based on the test results. If the error of the detection sensor exceeds a certain range, the sensor is calibrated. If the actuator cannot be controlled, it needs to be replaced to ensure the reliability and accuracy of the data.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] This invention provides intelligent control over the entire lifecycle of an air conditioning system, from the commissioning stage and trial operation stage to the stable intelligent control stage, comprehensively improving the control effect and operating efficiency of the air conditioning system.

[0020] By utilizing expert systems and reinforcement learning techniques, this invention generates high-quality datasets during the trial operation phase, providing effective data input for the stable intelligent control phase, thereby improving the accuracy of system control.

[0021] In the stable intelligent control stage, this invention adopts a combination of model-based control and reinforcement learning methods to adaptively adjust the control strategy and improve the control effect of the system under different operating conditions.

[0022] This invention continuously learns and optimizes the system online throughout its entire lifecycle to ensure continuous improvement in system performance, thereby meeting users' needs for energy efficiency and comfort. Attached Figure Description

[0023] Figure 1 A flowchart of the present invention. Detailed Implementation

[0024] The present invention will be further described below with reference to the accompanying drawings, but this is not intended to limit the present invention in any way. Any modifications or substitutions made based on the teachings of the present invention shall fall within the protection scope of the present invention.

[0025] This invention proposes an optimal data generation method for intelligent control throughout the lifecycle of HVAC systems, focusing on the proactive generation and management of data across the entire lifecycle. Specific technical means and measures are as follows:

[0026] 1) Lifecycle Phase Division: The lifecycle of the air conditioning system is divided into three phases: commissioning, trial operation, and stable intelligent control. Corresponding proactive data generation methods and algorithm adaptations are developed for each phase, taking into account its characteristics and requirements.

[0027] 2) Commissioning Phase: This phase fully utilizes its characteristics to generate basic, proactive data for the trial operation phase. This phase primarily employs an expert system to automatically detect, evaluate, and improve the data range and controllability of sensors and actuators, thus providing a solid foundation for the next phase of reinforcement learning and ensuring a smooth transition for the air conditioning system. An expert system is an intelligent computer program system containing a wealth of knowledge and experience at the level of an expert in a specific domain. It can apply artificial intelligence and computer technology to reason and judge based on the system's knowledge and experience, simulating the decision-making process of human experts to solve complex problems that require human intervention. The expert's domain knowledge must be represented in a specific way. Commonly used knowledge representation methods include production rules, semantic networks, frames, state spaces, logical schemas, scripts, procedures, and object-oriented programming. The expert system in this invention automatically detects, evaluates, and improves the data range and controllability of sensors and actuators in the HVAC system.

[0028] 3) Trial Operation Phase: The trial operation phase provides a high-quality dataset for the stable intelligent control phase. The expert system is responsible for providing positive feedback to the reinforcement learning system, assisting in generating a high-quality dataset for the reinforcement learning control of the HVAC system. This phase requires the data output from the debugging phase, and the reinforcement learning intelligent control system generates a control dataset at the optimal control frequency (1 MIN).

[0029] Reinforcement learning evolved from theories such as animal learning and adaptive control with parameter perturbations. Its basic principle is that if an agent's behavioral policy leads to a positive reward (reinforcement signal) from the environment, then the agent's tendency to produce this behavioral policy in the future will be strengthened. The agent's goal is to find the optimal policy in each discrete state to maximize the expected discounted reward.

[0030] Reinforcement learning views learning as an exploration and evaluation process. An agent selects an action to perform on the environment. After the environment accepts the action, its state changes, and a reinforcement signal (reward or punishment) is generated and fed back to the agent. Based on the reinforcement signal and the current state of the environment, the agent selects its next action, aiming to increase the probability of receiving positive reinforcement (reward). The selected action not only affects the immediate reinforcement value but also the environment's state at the next moment and the final reinforcement value.

[0031] Reinforcement learning differs from supervised learning in connectionist learning, primarily in the reinforcement signal. In reinforcement learning, the reinforcement signal provided by the environment is an evaluation (usually a scalar signal) of the agent's actions, rather than instructing the agent on how to produce the correct action. Because the external environment provides very little information, the agent must learn through its own experience. In this way, the agent acquires knowledge in an action-evaluation environment and improves its action plans to adapt to the environment.

[0032] The goal of reinforcement learning systems is to dynamically adjust parameters to maximize the reinforcement signal. If the gradient information r / A is known, supervised learning algorithms can be used directly. However, because the reinforcement signal r and the action A generated by the agent do not have a clear functional description, the gradient information r / A cannot be obtained. Therefore, reinforcement learning systems require some kind of random unit. Using this random unit, the agent searches the possible action space and finds the correct action. In this invention, the expert system feeds back the reward signal to the agent in reinforcement learning, enabling the reinforcement learning agent to continuously generate high-quality datasets of the HVAC system, laying the foundation for the next stage of control.

[0033] 4) Stable Intelligent Control Phase: A combined model-based and reinforcement learning intelligent control method is employed. During the first cooling and heating season of the HVAC system, reinforcement learning is used for intelligent control. From the second cooling and heating season onwards, the system switches to model-based control. If the system's operating data for a cooling or heating season is not in the historical database, the system automatically switches to reinforcement learning. The stable intelligent control phase primarily uses model-based control to improve control effectiveness, while reinforcement learning is used as a supplement for environmental sample generation and basic range control.

[0034] This invention uses K-means clustering to determine whether system running data is within the historical database. The core objective of K-Means is to divide a given dataset into K clusters, where K is a hyperparameter, and to provide the centroid of each sample data point. The specific steps consist of four steps:

[0035] (1) Data preprocessing. This mainly involves standardization and outlier filtering.

[0036] (2) Randomly select K centers, denoted as Kcenter.

[0037] (3) Define the loss function:

[0038] (4) Let t = 0, 1, 2, ... be the iteration step number, and repeat the following process until J converges:

[0039] (4.1) For each sample, assign it to the nearest center.

[0040]

[0041] (4.2) For each class center k, recalculate the center of that class.

[0042]

[0043] The core of KMeans is to first fix the centroids and adjust the class of each sample to reduce J; then, fix the class of each sample and adjust the centroids again to further reduce J. These two processes alternate, with J monotonically decreasing until it reaches a minimum value, at which point both the centroids and the class assignments converge. In some embodiments, the KMeans method is combined with swarm intelligence algorithms. The global search capability of swarm intelligence is used to select the initial clustering points for KMeans clustering. Alternatively, a combination of the Levy-Flight algorithm and Particle Swarm Optimization (PSO) can be used to find the global optimum.

[0044] 5) Lifecycle Integration: Seamless integration between different stages ensures a smooth transition for the air conditioning system. By providing a stable and reliable data source at the beginning of the lifecycle for later stable intelligent control, the effectiveness of intelligent control throughout the entire lifecycle is guaranteed.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] This invention provides intelligent control over the entire lifecycle of an air conditioning system, from the commissioning stage and trial operation stage to the stable intelligent control stage, comprehensively improving the control effect and operating efficiency of the air conditioning system.

[0047] By utilizing expert systems and reinforcement learning techniques, this invention generates high-quality datasets during the trial operation phase, providing effective data input for the stable intelligent control phase, thereby improving the accuracy of system control.

[0048] In the stable intelligent control stage, this invention adopts a combination of model-based control and reinforcement learning methods to adaptively adjust the control strategy and improve the control effect of the system under different operating conditions.

[0049] This invention continuously learns and optimizes the system online throughout its entire lifecycle to ensure continuous improvement in system performance, thereby meeting users' needs for energy efficiency and comfort.

[0050] In summary, compared with existing technologies, this invention has higher comprehensiveness, more efficient data generation capabilities, adaptive control strategies, and continuous optimization of system performance. It is also easier to implement and reproduce, bringing significant advantages and positive effects to the intelligent control of HVAC systems.

[0051] As used herein, the term "preferred" is meant as an example, illustration, or illustration. Any aspect or design described herein as "preferred" need not be construed as being more advantageous than other aspects or designs. Rather, the use of the term "preferred" is intended to present the concept in a specific manner. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusionary "or." That is, unless otherwise specified or clear from the context, "X uses A or B" naturally includes either of the permutations. That is, if X uses A; X uses B; or X uses both A and B, then "X uses A or B" is satisfied in any of the foregoing examples.

[0052] Furthermore, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art based on a reading and understanding of this specification and the accompanying drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the appended claims. In particular, with respect to the various functions performed by the aforementioned components (e.g., elements, etc.), the terminology used to describe such components is intended to correspond to any component (unless otherwise indicated) that performs the specified function of said component (e.g., is functionally equivalent to it), even if structurally not equivalent to the disclosed structure performing the functions in the exemplary implementations of this disclosure shown herein. Moreover, although specific features of this disclosure have been disclosed with respect to only one of several implementations, such features may be combined with one or more features of other implementations that may be desirable and advantageous for a given or particular application. Furthermore, with regard to the use of the terms “comprising,” “having,” “containing,” or variations thereof in the Detailed Description or claims, such terms are intended to be included in a manner similar to the term “including.”

[0053] The functional units in this invention embodiment can be integrated into a processing module, or each unit can exist physically separately, or multiple units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. The aforementioned devices or systems can execute the storage methods in the corresponding method embodiments.

[0054] In summary, the above embodiments are one implementation of the present invention, but the implementation of the present invention is not limited to the embodiments described above. Any changes, modifications, substitutions, combinations, or simplifications made that deviate from the spirit and principle of the present invention should be considered equivalent substitutions and are included within the protection scope of the present invention.

Claims

1. A method for proactively generating data throughout the optimal intelligent control lifecycle of HVAC systems, characterized in that: Includes the following steps: Commissioning phase: Generate basic active data for the trial operation phase, including: testing all sensors and actuators of the air conditioning system; using an expert system for automatic testing to evaluate the data range, controllability and other key parameters of the sensors and actuators; Trial operation phase: To provide high-quality datasets for the stable intelligent control phase, the expert system is responsible for providing positive feedback to the reinforcement learning to assist the reinforcement learning control of the HVAC system in generating high-quality datasets; This phase uses the data output from the debugging phase, and the reinforcement learning intelligent control system generates control datasets at the optimal control frequency; Stable intelligent control phase: The intelligent control method combining model and reinforcement learning is adopted. During the first cooling and heating season of the HVAC system, reinforcement learning is used for intelligent control. Starting from the second cooling and heating season, the model control method is switched to the model control method. K-means clustering is used to determine whether the system's operating data for a cooling and heating season is in the historical database. When the system's operating data for a cooling and heating season is not in the historical database, the system automatically switches to reinforcement learning for control. The K-means clustering method for determining whether system runtime data is in the historical database specifically includes: Preprocessing of operating data for HVAC systems, including standardization and outlier filtering; K centers are randomly selected, denoted as Kcenter. ; Define the loss function: ; Let t = 0, 1, 2, ... be the number of iterations, and repeat the following process until the loss function J converges: For each sample, assign it to the nearest center. For each class center k, the center of that class is recalculated. 。 2. The method for actively generating data within the optimal intelligent control lifecycle of HVAC systems according to claim 1, characterized in that, During the debugging phase, the system is adjusted and optimized as necessary based on the test results. If the sensor error exceeds a certain range, the sensor is calibrated. If the actuator cannot be controlled, it needs to be replaced to ensure the reliability and accuracy of the data. The trial operation phase uses the data output from the debugging phase. Smooth transitions between different stages ensure a seamless transition for the air conditioning system. By providing a stable and reliable data source for later intelligent control in the early stages of the life cycle, the system ensures effective intelligent control throughout its entire life cycle.

3. The method for actively generating data within the optimal intelligent control lifecycle of HVAC systems according to claim 1, characterized in that, The stable intelligent control stage primarily uses model-based control to improve control effectiveness, supplemented by reinforcement learning for environmental sample generation and basic range control.

4. The method for actively generating data within the optimal intelligent control lifecycle of HVAC systems according to claim 1, characterized in that, The expert system is a set of operating parameters of the HVAC system, while the reinforcement learning system controls the outlet water temperature of the chiller unit within a certain range and continuously generates a dataset within that range.

Citation Information

Patent Citations

  • Intelligent diagnosis method based on full life cycle of air conditioner room

    CN115183389A

  • Life cycle cost calculation system of architectural structure

    JP2003336311A