Environment controller and method for generating a predictive model of a neural network through distributed reinforcement learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current environment control systems in buildings face challenges in efficiently generating and improving predictive models for neural networks, requiring a large number of samples during the training phase and lacking automation for improving the models during the operational phase.
Innovation Solution
An environment controller and method utilizing distributed reinforcement learning, where the controller determines environmental characteristics, receives set points, executes a neural network inference engine, modifies commands, and transmits metrics to a training server, which updates the predictive model weights, enabling continuous improvement of the model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large number of samples are collected during the training phase, then the accuracy of the predictive model is improved, but the time required for training increases significantly
Solution Approach 1:
The system performs preliminary actions by collecting and storing environmental data, commands, and metrics during the operational phase before formal training is needed. This pre-collection of training samples during normal operation reduces the time required for subsequent model training while maintaining high accuracy through distributed data gathering from multiple sources
Solution Approach 2:
The system enables continuous collection of training data during the operational phase through distributed environment controllers that constantly monitor and record environmental characteristics, commands executed, and performance metrics. This continuous data accumulation allows the predictive model to be trained on large datasets without requiring dedicated offline training periods, thus improving accuracy while minimizing time loss
2Measurement precision
If manual generation of training samples is performed, then the quality of training data can be controlled, but the productivity of the training process decreases
Solution Approach 1:
The system implements self-service by automatically generating training samples through environment controllers that autonomously collect environmental data, record executed commands, and capture performance metrics during normal operation. This automated self-collection process eliminates manual intervention while maintaining high data quality through structured data generation protocols, thereby significantly improving training productivity
Solution Approach 2:
The system uses feedback mechanisms where environment controllers transmit metrics representing command execution results back to the training system. This feedback loop ensures that only high-quality, relevant training samples are generated and used, maintaining training data quality while automating the entire sample generation process to improve productivity
3Adaptability or versatility
If the predictive model is updated frequently, then the adaptability of the system to changing conditions is improved, but the computational resources required increase
Solution Approach 1:
The system applies partial action by updating the predictive model selectively rather than continuously. Training samples are accumulated over time and used to update the model at optimized intervals, performing just enough training to maintain adaptability without excessive computational resource consumption that would result from frequent continuous training
Solution Approach 2:
The system segments the training process by distributing data collection across multiple environment controllers that independently gather and pre-process training samples locally. This segmentation allows parallel data collection and preparation, reducing the computational burden on any single system when model updates are performed, thus maintaining adaptability while managing resource consumption
Data Source
AI summary
Interactions between a training server and a plurality of environment controllers are used for updating the weights of a predictive model used by a neural network executed by the plurality of environment controllers. Each environment controller executes the neural network using a current version of the predictive model to generate outputs based on inputs, modifies the outputs, and generates metrics representative of the effectiveness of the modified outputs for controlling the environment. The training server collects the inputs, the corresponding modified outputs, and the corresponding metrics from the plurality of environment controllers. The collected inputs, modified outputs and metrics are used by the training server for updating the weights of the current predictive model through reinforcement learning. A new predictive model comprising the updated weights is transmitted to the environment controllers to be used in place of the current predictive model.