Reinforcement Learning Equipment Control With Evaluation-Model Rewards
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems lack an efficient method for autonomously controlling complex equipment using AI to identify operational bottlenecks and improve performance based on real-time data analysis.
Innovation Solution
An operation system utilizing an evaluation model generated by machine learning and an operation model by reinforcement learning to control equipment, where the output of the evaluation model serves as a reward in the reinforcement learning process, enabling autonomous control and continuous improvement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If reinforcement learning is used to generate an operation model with the evaluation model's output as reward, then autonomous control capability is improved, but system complexity increases
Solution Approach 1:
The system is divided into two distinct models: an evaluation model that assesses equipment states and an operation model that generates control actions. This segmentation allows each model to specialize in one function, making the overall complex system manageable through modular design where the evaluation model outputs serve as rewards for the operation model's reinforcement learning process
Solution Approach 2:
The evaluation model acts as an intermediary between the equipment state and the operation model. Instead of the operation model directly processing complex equipment states, it receives simplified evaluation scores as rewards, which mediate the learning process and reduce the complexity of the reinforcement learning problem
2Productivity
If continuous learning and updating of models is performed, then equipment performance improvement is achieved, but computational resources and time are consumed
Solution Approach 1:
The system implements continuous learning where the operation model is repeatedly updated through reinforcement learning using the evaluation model's outputs as rewards. This continuous action allows the system to progressively improve equipment performance by constantly adapting to new operational data and conditions without interruption
Solution Approach 2:
The evaluation model provides continuous feedback to the operation model by generating reward signals based on equipment state assessments. This feedback mechanism enables the operation model to learn from its actions and continuously improve its control strategy, creating a closed-loop system that automatically optimizes performance over time
Data Source
AI summary
Provided is an operation system including: an evaluation model generation apparatus configured to generate, by machine learning, an evaluation model configured to output an indicator indicating a result of evaluating a state in a piece of equipment with respect to an intended target based on an operation target in the piece of equipment and a state in the piece of equipment; an operation model generation apparatus configured to generate an operation model configured to output an action corresponding to the state in the piece of equipment, by reinforcement learning in which an output of the evaluation model is set as at least a part of a reward; and a control apparatus configured to apply, to a controlled object in the piece of equipment, a manipulated variable based on the action that is output by the operation model according to the state in the piece of equipment.