Neural Network Demand Response Control for Energy Distribution Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing demand response systems face challenges in controlling energy-constrained sources like Electric Vehicles and HVAC systems due to high dimensionality and scalability issues in state space, requiring complex system models that are difficult to maintain and adapt to changes, and lack of visibility into end-to-end infrastructure dynamics.
Innovation Solution
A model-free reinforcement learning approach using a convolutional neural network to extract spatiotemporal features from historical observations aggregated in 2D grid structures, combined with fully connected neural networks to determine control actions, allowing end-to-end visibility and reducing computational intensity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If model-based control techniques are used to control demand flexibility of energy-constrained sources, then domain knowledge can be incorporated directly in the model, but the model needs to be accurate, tuned and maintained which increases device complexity and reduces adaptability
Solution Approach 1:
The patent replaces traditional model-based control mechanisms with a neural network-based reinforcement learning system. Instead of using explicit mathematical models that require manual construction and maintenance, the system uses neural networks to learn control policies directly from system interactions, eliminating the need for manual model tuning and adaptation while maintaining domain knowledge through learned representations
Solution Approach 2:
The neural network controller learns autonomously through reinforcement learning by interacting with the demand response system. The system self-adjusts and adapts to changes without requiring external intervention for model re-tuning or re-construction, as the neural network continuously learns optimal control strategies from system feedback
2Adaptability or versatility
If model-free reinforcement learning is used with high dimensionality state space, then model maintenance is eliminated, but dimensionality and scalability issues arise increasing device complexity
Solution Approach 1:
The patent segments the high-dimensional state space by grouping energy-constrained flexibility sources into clusters based on similar characteristics and behaviors. This clustering approach reduces the effective state space dimensionality while preserving the essential dynamics, allowing the reinforcement learning algorithm to scale to large systems without prohibitive computational complexity
Solution Approach 2:
The patent transforms the high-dimensional state representation into a more manageable form by introducing cluster-based aggregation dimensions. Instead of treating each individual resource separately, the system aggregates resources into clusters, effectively reducing dimensionality while maintaining the ability to capture system-wide dynamics through the cluster-level state representations
3Device complexity
If cluster control is implemented for large systems, then dimensionality is reduced through aggregation, but end-to-end visibility into infrastructure dynamics is lost
Solution Approach 1:
The patent implements a nested control architecture where cluster-level controllers operate at one level and individual resource-level controllers operate at another level within the same system. This nested structure allows the system to benefit from dimensionality reduction at the cluster level while maintaining detailed visibility and control at the individual resource level, effectively preserving end-to-end system dynamics information across multiple hierarchical levels
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
A deep approximation neural network architecture is described which extrapolates data over unseen states for demand response applications in order to control distribution systems like product distribution systems of which energy distribution systems, e.g. heat or electrical power distribution, are one example. The present invention describes a model-free control technique mainly in the form of Reinforcement Learning (RL) whereby a controller learns from interaction with the system to be controlled to control product distribution s of which energy distribution systems, e.g. heat or electrical power distribution, are one example.