Automated Action Label Set Reduction for RL Planning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The large size of planning action label sets in relational Markov Decision Processes (MDPs) hinders efficient learning in Reinforcement Learning (RL) for planning problems, as they require manual reduction and lead to inefficient computations and data hunger.
Innovation Solution
An automated method for reducing action label sets by creating an initial label set, performing machine logic-based label reduction, and recasting the planning problem as a Markov decision process using the reduced label set, leveraging mutex groups and operator parameter reduction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a complete action label set is used in relational MDPs, then the planning domain is fully represented, but the learning efficiency deteriorates due to large action space size
Solution Approach 1:
The patent extracts and removes redundant action labels from the complete action label set by identifying mutex groups (mutually exclusive operators). Only non-redundant labels are retained in the reduced action space, eliminating unnecessary actions while preserving the essential planning domain representation.
Solution Approach 2:
The patent changes the parameter of action space size by transforming the complete label set into a reduced label set. This parameter transformation maintains the necessary planning information while significantly reducing the action space dimensionality, thereby improving learning efficiency.
2Productivity
If manual reduction of action label sets is performed, then the action space size is reduced, but the process becomes infeasible for large planning domains
Solution Approach 1:
The system performs self-service by automatically identifying mutex groups and reducing the action label set without human intervention. The algorithm autonomously analyzes operator relationships, determines mutual exclusivity, and generates the reduced action space, making the process feasible for large planning domains.
Solution Approach 2:
The patent replaces the manual mechanical process of action label reduction with an automated computational system. The machine logic systematically identifies mutex groups and performs reduction, substituting human effort with an efficient algorithmic approach that scales to large domains.
3Adaptability or versatility
If Reinforcement Learning is applied to planning problems, then data hunger increases and computational efficiency decreases, but no symbolic model is required
Solution Approach 1:
The patent segments the action space into mutex groups, where operators within each group are mutually exclusive. This segmentation allows the RL agent to learn from fewer actions by recognizing that only one operator from each mutex group can be applied at any given state, reducing the effective action space and data requirements.
Data Source
AI summary
Technology for: (i) receiving a domain-dependent artificial intelligence planning problem including definitions for a plurality of operators; (ii) creating an initial version of a label set, which defines an initial version of an action space, with the label set including a plurality of labels, and with each label of the plurality of labels respectively corresponding to the operators of the plurality of operators; (iii) performing, automatically and by machine logic, a label reduction on the initial version of the label set to obtain a reduced version of the label set that defines a reduced action space; and (iv) recasting the artificial planning problem as a first Markov decision process using the reduced version of label set.