HVAC Supervisory Control Using DRL and Rule-Based Conflict Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing HVAC control systems face challenges in optimizing energy consumption and occupant comfort due to conflicts in rule-based systems and the lengthy learning phase of machine learning-based approaches, particularly in resolving conflicting intuitions and rules.
Innovation Solution
A method and system that combines Deep Reinforcement Learning (DRL) with domain knowledge using Expressive Decision Tables (EDT) to analyze HVAC parameters and select optimal control actions, resolving conflicts by maximizing cumulative rewards and reducing learning duration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If rule-based systems are used for HVAC control, then ease of implementation is improved, but energy optimization and conflict resolution capability deteriorate
Solution Approach 1:
The system segments the control architecture into two distinct modules: a rule-based decision table engine that handles interpretable control logic, and a DRL agent that handles optimization and conflict resolution. This segmentation allows each component to specialize - the rule engine provides ease of implementation while the DRL agent provides energy optimization capability.
Solution Approach 2:
The system merges rule-based control and machine learning-based control into a unified hybrid architecture. The rule-based system provides structured decision-making while the DRL agent learns optimal policies from data, combining the interpretability of rules with the adaptability of learning to achieve both ease of implementation and energy optimization.
2Productivity
If machine learning-based approaches are used for HVAC control, then energy optimization is improved, but learning duration and implementation complexity deteriorate
Solution Approach 1:
The system performs preliminary action by pre-training the DRL agent offline using historical building data and simulated environments. This offline pre-training allows the agent to learn optimal control policies before deployment, significantly reducing the online learning duration and enabling rapid adaptation to real-time conditions without lengthy training periods.
Solution Approach 2:
The rule-based decision table acts as an intermediary that guides the DRL agent's learning process. By providing structured domain knowledge and initial control policies, the intermediary reduces the exploration space the DRL agent must search, thereby accelerating convergence and reducing learning duration while maintaining energy optimization capabilities.
3Adaptability or versatility
If multiple rules are used in rule-based systems, then control coverage is improved, but rule conflicts and decision-making complexity deteriorate
Solution Approach 1:
The system implements feedback mechanisms where the DRL agent continuously monitors the performance and conflicts of multiple rules. When rule conflicts are detected, the DRL agent receives feedback about the conflicting decisions and learns to resolve them by adjusting its policy, thereby maintaining comprehensive control coverage while reducing decision-making complexity through learned conflict resolution strategies.
Solution Approach 2:
The DRL agent serves as an intermediary layer between multiple conflicting rules and the final control decision. It receives inputs from multiple rules, evaluates their compatibility and optimality, and mediates to select the best action, thereby enabling comprehensive control coverage while managing rule conflicts systematically rather than allowing direct conflicts between rules.
Data Source
Figure 1A
Figure 1B
Figure 2A
AI summary
HVAC control system's supervisory control is crucial for energy-efficient thermal comfort in buildings. The control logic is usually specified as 'if-then-that-else' rules that capture the domain expertise of HVAC operators, but they often have conflict that may lead to sub-optimal HVAC performance. Embodiments of the present disclosure provide a method and system for optimized Heating, ventilation, and air-conditioning (HVAC) control using domain knowledge combined with Deep Reinforcement Learning (DRL). The system disclosed utilizes Deep Reinforcement Learning (DRL) for conflict resolution in a HVAC control in combination with domain knowledge in form of control logic. The domain knowledge is predefined in an Expressive Decision Tables (EDT) engine via a formal requirement specifier consumable by the EDT engine to capture domain knowledge of a building for the HVAC control.