Building Control Using Reinforcement Learning for Peak Demand
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Building automation systems face challenges in efficiently regulating environmental factors like temperature and humidity due to external factors, leading to suboptimal performance and increased energy consumption, especially with time-varying electricity prices and demand charges.
Innovation Solution
A building management system that uses reinforcement learning models, specifically a Deep Q-Network (DQN) and double DQN algorithms, to optimize zone-air temperature setpoints based on historical data, time-of-use pricing, and comfort bounds, minimizing energy costs and peak demand charges while maintaining thermal comfort.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If building automation systems use traditional setpoint-based control, then the system operation is simple and easy to understand, but the system cannot adapt to external factors and time-varying electricity prices, resulting in suboptimal performance and increased energy consumption
Solution Approach 1:
The system employs reinforcement learning models that automatically learn and optimize control policies without human intervention. The Deep Q-Network and double DQN algorithms self-adjust to external factors and electricity price variations, enabling the system to serve itself in adapting to changing conditions while maintaining reasonable complexity through automated learning rather than manual reconfiguration
Solution Approach 2:
The system dynamically changes control parameters (zone-air temperature setpoints) based on learned patterns from historical data and real-time conditions. The reinforcement learning models continuously adjust temperature setpoints according to external factors and electricity pricing, transforming the static setpoint-based control into a dynamic parameter-adjustment system that adapts to varying conditions
2Loss of energy
If the system pre-cools buildings during off-peak hours to reduce peak demand, then energy costs and peak demand charges are reduced, but the system complexity increases due to predictive control requirements
Solution Approach 1:
The system performs preliminary cooling actions during off-peak hours by lowering zone-air temperature setpoints when electricity rates are lower. The reinforcement learning models predict future peak demand periods and proactively cool buildings in advance, storing thermal energy in building thermal mass to reduce the need for intensive cooling during peak hours, thereby reducing both energy costs and peak demand charges
Solution Approach 2:
The system implements feedback mechanisms where the reinforcement learning models continuously monitor actual temperature deviations, energy consumption, and electricity price signals. The double DQN algorithm uses feedback from previous control actions and observed outcomes to refine future pre-cooling strategies, optimizing the balance between preliminary cooling actions and energy cost reduction while adapting to actual building response
3Loss of energy
If the system uses reinforcement learning models to optimize control policies, then energy cost savings of around 6% are achieved, but the measurement and training requirements become more difficult
Solution Approach 1:
The system performs preliminary training of reinforcement learning models using historical building operation data before deployment. The Double DQN algorithm is trained offline on past temperature, humidity, and electricity price data to learn optimal control policies. This preliminary training phase allows the model to accumulate learning from historical patterns without affecting real-time building operation, separating the difficult measurement and training requirements from ongoing operational complexity
Solution Approach 2:
The system uses simulated building environments as intermediaries to train and validate reinforcement learning models before deploying them to actual buildings. The simulation environment provides a safe testing ground where the Double DQN algorithm can be trained on virtual building data, allowing extensive experimentation and model refinement without the complexity of directly measuring and analyzing real building performance during the training phase
Data Source
AI summary
Sensorized commercial buildings are a rich target for building a new class of applications that improve operational and energy efficiency of building operations that take into account human activities. Such applications, however, rarely experience widespread adoption due to the lack of a common descriptive schema that would enable porting these applications and systems to different buildings. Our demo presents Brick [4], a uniform schema for representing metadata in buildings. Our schema defines a concrete ontology for sensors, subsystems and relationships among them, which enables portable applications. Using a web application, we will demonstrate real buildings that have been mapped to the Brick schema, and show application queries that extracts relevant metadata from these buildings. The attendees would be able to create example buildings and write their own queries.