Hierarchical Workflow Planning With Simulation-Guided Debugging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hierarchical planning approaches for robotic systems are complex and require expert knowledge for debugging, as they lack intuitive visualization and feedback, making it difficult for non-experts to identify and resolve issues such as collisions or task failures in workflow processes.
Innovation Solution
A method and system that utilize a Graphical User Interface (GUI) for visualizing hierarchical plans, allowing users to select and execute debugging operators to identify and correct failures, combined with a simulation environment for real-time monitoring and updating of plans to achieve intermediate goals, using Large Language Models for operator generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If hierarchical planning is used to reduce complexity, then planning feasibility is improved, but debugging difficulty increases due to lack of information at different hierarchy levels
Solution Approach 1:
The planning system is divided into multiple hierarchy levels (high-level task planning, mid-level sequence planning, low-level motion planning). Each level handles specific aspects of the planning problem, reducing the complexity at each individual level while maintaining overall system capability. Debugging can be performed at each level independently, focusing on specific aspects rather than the entire system.
Solution Approach 2:
The system implements feedback mechanisms that provide debugging information across hierarchy levels. When a planning failure occurs, the system traces and reports the failure back through the hierarchy, providing users with information about what went wrong at each level. This feedback loop enables effective debugging despite the hierarchical structure.
2Reliability
If traditional debugging methods are used with standard programming IDEs, then execution checking is possible, but time consumption increases due to manual checking of numerous possibilities
Solution Approach 1:
The system performs self-diagnosis and self-debugging by automatically tracing planning failures, identifying the root cause, and suggesting corrections. Instead of requiring users to manually check numerous possibilities, the system autonomously analyzes the failure, provides diagnostic information, and can even automatically repair the plan, significantly reducing debugging time.
Solution Approach 2:
The patent replaces manual mechanical debugging processes (step-by-step manual checking in IDEs) with automated computational debugging. The system uses algorithms to automatically trace failures, analyze causes, and suggest fixes, substituting human manual effort with automated intelligent systems.
3Device complexity
If no visualization interface is provided, then system simplicity is maintained, but user understanding and debugging capability deteriorate
Solution Approach 1:
The system introduces visualization interfaces and debugging tools as intermediaries between the complex hierarchical planning system and the user. These intermediaries translate internal planning states, failures, and diagnostic information into user-friendly visual representations, enabling users to understand and debug the system without directly interacting with the complex underlying mechanics.
Data Source
Figure 1A~1B
Figure 2
Figure 3
AI summary
Embodiments of the present disclosure provide a method and system for generating and debugging hierarchical plans for a workflow process. The method may comprise providing a subset of operators from a plurality of operators representing possible operations one or more machines are able to perform during an execution of the workflow process; providing a goal state definition for the workflow process; generating, by one or more processors of a planning engine and based on the one or more operators and the goal state definition, a hierarchical plan including one or more hierarchical levels of the workflow process; deploying and executing the hierarchical plan on the one or more machines and/or in one or more simulated digital representations of the one or more machines and of an environment where the one or more machines are located; in response to at least one failed operator occurring during execution of a respective hierarchical level of the hierarchical plan, providing one or more debugging operators associated with the at least one failed operator, the one or more debugging operators including feasible operators performable by the one or more machines in the respective hierarchical level of the hierarchical plan where the at least one failed operator is located, wherein the feasible operators ensure that an intermediate goal state is achieved for the one or more machines and/or the simulated digital representations for the respective hierarchical level; while iteratively executing the one or more feasible operators, continuously monitoring the intermediate goal state of the one or more machines and/or of the simulated digital representations and if the intermediate state reaches an intermediate goal at the respective hierarchical level, updating the hierarchical plan by replacing the at least one failed operator with the one or more debugging operators leading to the intermediate goal state of the one or more machines and/or of the simulated digital representations; and executing the updated hierarchical plan until the goal state is achieved.