Optimized HVAC control using domain knowledge combined with deep reinforcement learning (DRL)
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing HVAC control systems face challenges in optimizing energy consumption and occupant comfort due to conflicts in rule-based systems and the lengthy exploration phase of machine learning-based approaches like DRL.
Innovation Solution
A method and system that combines domain knowledge with Deep Reinforcement Learning (DRL) using Expressive Decision Tables (EDT) to analyze HVAC parameters and resolve conflicts, thereby selecting optimal control actions that maximize cumulative rewards.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If rule-based systems are used for HVAC control, then ease of implementation is improved, but energy optimization and conflict resolution capability deteriorate
Solution Approach 1:
The patent introduces a conflict resolution module as an intermediary between the rule-based system and the HVAC controller. This module receives control actions from multiple conflicting rules, resolves their conflicts using prioritization strategies, and outputs a single optimized control action. This intermediary layer maintains the ease of rule-based implementation while adding sophisticated conflict resolution capability to improve energy optimization.
2Productivity
If DRL-based approaches are used for HVAC control, then optimal control performance is improved, but learning duration and implementation complexity worsen
Solution Approach 1:
The patent applies preliminary action by pre-training the DRL agent using synthetic data generated from building simulations before deploying it to real HVAC systems. This pre-training phase allows the agent to learn optimal control strategies in a virtual environment, significantly reducing the online learning duration required when the system is deployed to actual buildings. The pre-trained policy serves as a good initial starting point for fine-tuning with real-world data.
3Adaptability or versatility
If multiple rules are applied in rule-based systems, then control coverage is improved, but conflict resolution capability worsens
Solution Approach 1:
The patent segments the conflict resolution process into distinct modular components: rule evaluation module, conflict detection module, prioritization module, and action selection module. Each module handles a specific aspect of conflict resolution independently. This segmentation allows the system to manage multiple rules effectively while keeping the complexity of conflict resolution tractable through modular design, where each segment can be developed and tested independently.
Data Source
AI summary
HVAC control system's supervisory control is crucial for energy-efficient thermal comfort in buildings. The control logic is usually specified as ‘if-then-that-else’ rules that capture the domain expertise of HVAC operators, but they often have conflict that may lead to sub-optimal HVAC performance. Embodiments of the present disclosure provide a method and system for optimized Heating, ventilation, and air-conditioning (HVAC) control using domain knowledge combined with Deep Reinforcement Learning (DRL). The system disclosed utilizes Deep Reinforcement Learning (DRL) for conflict resolution in a HVAC control in combination with domain knowledge in form of control logic. The domain knowledge is predefined in an Expressive Decision Tables (EDT) engine via a formal requirement specifier consumable by the EDT engine to capture domain knowledge of a building for the HVAC control.