Control Target Device Selection via Disturbance Extraction
Solution Overview
Problem
Conventional reinforcement learning systems fail to consider disturbances in external conditions and require manual specification of factors affecting reward variations, leading to inefficient operation of devices not involved in reward achievement, which can reduce overall system efficiency.
Innovation Solution
A control target device selection apparatus that automatically extracts disturbance components, defines situations based on these components, and updates the learning model to optimize device control values, thereby identifying and excluding non-involved devices from the control process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual specification of external factors is used in reinforcement learning, then the system can control devices based on predefined situations, but the device complexity and operational burden increase due to manual configuration requirements
Solution Approach 1:
The system automatically extracts disturbance component factors from environmental data without manual specification. The learning model autonomously identifies which external factors affect the reward and creates situations based on these factors, eliminating the need for manual configuration while maintaining controlled device operation.
Solution Approach 2:
The system dynamically changes the parameter of situation definition from manual predefined categories to automatically extracted disturbance components. By transforming external factor data into situation classifications based on actual reward-affecting patterns, the system adapts to varying operational conditions without requiring manual reconfiguration.
2Productivity
If all devices operate continuously in reinforcement learning systems, then the system maintains readiness to respond to changes, but device operation efficiency decreases due to non-involved devices consuming resources
Solution Approach 1:
The system segments device operation into involved and non-involved devices based on their contribution to reward achievement. By classifying devices according to whether they affect the reward under current situations, the system can selectively operate only those devices that are actually needed, improving overall productivity while reducing energy waste from continuously operating irrelevant devices.
Solution Approach 2:
The system dynamically determines which devices should be operated based on current situations and reward structures. Rather than fixed continuous operation, the device selection adapts to changing environmental conditions and reward requirements, allowing the system to optimize the balance between readiness and energy efficiency in real-time.
3Adaptability or versatility
If the learning model is updated frequently to adapt to changing conditions, then the system maintains high adaptability, but the computational load and processing time increase
Solution Approach 1:
The system performs preliminary extraction of disturbance component factors and pre-definition of situations based on environmental data before actual device control begins. By preparing the situation classifications and identifying relevant external factors in advance, the system reduces the computational burden during real-time operation and minimizes the time needed for frequent updates.
Solution Approach 2:
The system extracts only the essential disturbance component factors from the environmental data that actually affect the reward, rather than processing all available data. This selective extraction approach reduces the computational complexity of model updates while maintaining adaptability to the most important changing conditions.
Data Source
AI summary
A control target device selection apparatus (1) includes: a situation classification unit (122) for extracting an external factor affecting a reward as a component and defining a situation for controlling a control target device (5) as a classification for each divided range; a learning data management unit (123) for storing learning data for each device control factor pattern in a learning data DB; a learning model management unit (124) for generating a learning model for each classification, a non-involved device specification unit (1271) for determining non-involved devices and a range of classification of non-involved devices; and, with respect to the range of the non-involved classification, a device control unit (130) for transmitting a device control value to each control target device (5) excluding the non-involved device.