Hybrid Optimal Control Using Reinforcement Learning and Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing control methods, such as PID control, struggle to effectively integrate feedback-based control with reinforcement learning, especially when encountering new data types, and reinforcement learning techniques often fail to adapt to novel data.
Innovation Solution
A method that combines reinforcement learning with feedback control using a neural network, where state information is processed through a reinforcement learning control model and a feedback control algorithm to calculate optimal control information, leveraging a confidence score and weighted sum operations to determine the optimal control.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If reinforcement learning technique is used for control, then adaptability to new data types is improved, but reliability of control operation deteriorates due to dependence on training data
Solution Approach 1:
The patent combines reinforcement learning control model with feedback control algorithm to create a hybrid control system. The reinforcement learning model provides adaptability to new data types while the feedback control algorithm ensures reliability by using error-based control. The two control approaches are integrated to mutually complement their strengths and weaknesses.
Solution Approach 2:
The control system uses a composite structure combining two different control methodologies (reinforcement learning and feedback control) similar to how composite materials combine different materials to achieve superior properties. This hybrid approach allows the system to leverage the adaptability of reinforcement learning while maintaining the reliability of traditional feedback control.
2Reliability
If feedback control algorithm is used, then reliability of control operation is improved, but adaptability to new data types deteriorates
Solution Approach 1:
The patent integrates feedback control algorithm with reinforcement learning control model to create a hybrid system. The feedback control provides reliable error-based control while the reinforcement learning component adds adaptability to new data types, resolving the trade-off between reliability and adaptability.
3Reliability
If hybrid control system combining reinforcement learning and feedback control is used, then adaptability and reliability are improved, but device complexity increases
Solution Approach 1:
The patent merges reinforcement learning and feedback control into a unified hybrid control system that achieves both adaptability and reliability. While this increases system complexity, it resolves the fundamental trade-off between adaptability and reliability that exists when using either control method alone.
Data Source
AI summary
According to an exemplary embodiment of the present disclosure, an optimal control method performed by a computing device including at least one processor is disclosed. The method includes acquiring state information including at least one state variable; calculating first control information by inputting the state information to a reinforcement learning control model; calculating second control information from the state information based on a feedback control algorithm; and calculating optimal control information based on the first control information and the second control information.