Interpretable Policy Creation Using Rule-Based Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The reinforcement learning method used to express know-how in processing plants is difficult for users to understand due to its reliance on neural networks, making it challenging to transfer high-quality work procedures effectively.
Innovation Solution
A policy creation apparatus that generates policies with a combination of rules for determining object states and actions, including action determination, policy evaluation, and selection, allowing for high-quality and visible policy creation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If reinforcement learning method with neural network is used to express know-how, then policy quality is improved, but user understanding and visibility deteriorate
Solution Approach 1:
The patent segments the neural network policy into multiple interpretable rules, each representing a specific condition-action pair. This segmentation allows the system to maintain high policy quality through neural network learning while improving user understanding by presenting discrete, explainable rules instead of a monolithic black-box model.
Solution Approach 2:
The patent introduces an intermediary layer that translates neural network outputs into human-readable rules. This intermediary mechanism bridges the gap between the high-performance neural network representation and the user-friendly rule-based format, enabling both high policy quality and good user understanding.
2Adaptability or versatility
If neural network model is used to represent policies, then learning capability is improved, but model interpretability deteriorates
Solution Approach 1:
The patent divides the neural network's decision-making process into segmented rules that can be individually interpreted. Each rule captures a specific aspect of the learned policy, preserving the learning capability of the neural network while recovering interpretability through structured rule representation.
Solution Approach 2:
The patent changes the parameter representation from continuous neural network weights to discrete rule parameters with explicit conditions and actions. This parameter transformation maintains the expressive power needed for learning while making the model interpretable through human-understandable parameter meanings.
Data Source
AI summary
A policy creation unit creates a plurality of policies each including a predetermined number of rules in which a condition for determining a state of an object is combined with an action in the state. An action determination unit determines, for each of the plurality of created policies, an action to be performed while a state of the object is changed from a first state to a second state in accordance with the policy. A policy evaluation unit determines, in accordance with a difference between a state between the first state and the second state and a desired state, the quality of the determined action and determines, for each of the plurality of policies, the quality of the policy based on the determined quality of the action. A policy selection unit selects the policy that is determined to have a high quality.