Operation Planning Policy for Adaptive Heuristic Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Heuristic performance varies across domains in planning systems, making it difficult to combine inadmissible heuristic estimates effectively, which can negatively impact search processes, especially in satisficing planning where inaccurate estimates can hinder progress.
Innovation Solution
A policy trained via Dynamic Algorithm Configuration (DAC) using reinforcement learning is employed for dynamic heuristic selection, taking into account internal search dynamics to optimize heuristic choice based on current state information and costs, thereby improving search efficiency and coverage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple admissible heuristic estimates are combined using maximum in optimal planning, then the heuristic performance is improved, but in satisficing planning the combination of inadmissible heuristic estimates becomes difficult and may have negative effects
Solution Approach 1:
The patent applies dynamics by transitioning from static heuristic combination methods to a dynamic approach where a reinforcement learning agent selects heuristics adaptively based on the current search state. The policy learns to dynamically choose which heuristic to apply in each situation, making the system responsive to changing search dynamics rather than using fixed combination rules.
Solution Approach 2:
The patent changes the parameter of heuristic selection from fixed combination weights to dynamically adjusted selections based on search state. The reinforcement learning agent modifies which heuristic is active based on observed search dynamics, effectively changing the heuristic parameter adaptively rather than relying on static combination methods.
2Productivity
If classical heuristic search methods are used, then the search process is simple to implement, but the convergence speed and solution quality are limited
Solution Approach 1:
The patent applies preliminary action by pre-training a reinforcement learning policy on search dynamics before actual planning execution. This pre-trained policy is then deployed to guide heuristic selection during search, allowing the system to benefit from learned expertise without adding computational overhead during the actual planning process.
Solution Approach 2:
The patent introduces an intermediary reinforcement learning agent that mediates between the search state and heuristic selection. This intermediary learns the optimal mapping from search dynamics to heuristic choices, acting as a bridge that translates search state information into effective heuristic selections without requiring direct complex interactions between multiple heuristics.
3Adaptability or versatility
If a single heuristic is used throughout the search, then the system is simple to manage, but the adaptability to different search situations is reduced
Solution Approach 1:
The patent applies self-service by enabling the reinforcement learning agent to autonomously learn and select heuristics based on observed search dynamics. The system serves itself by automatically adapting heuristic selection without requiring manual configuration or intervention, learning the optimal policy through interaction with the search process.
Data Source
AI summary
A computer-implemented method for planning an operation of a technical system within its environment. The method includes: obtaining state information comprising: a current domain, a time step and a current state; determining by heuristics costs for reachable states from the current state; selecting a heuristics by a policy out of a set of predefined heuristics depending on the state information and costs; choosing the state with the lowest cost returned by the selected heuristic from the reachable states, and determining an operation of the technical system out of the set of possible operation that has to be carried out by the technical system to reach said state with the lowest costreturned by the selected heuristic.