Robotic Task Anticipation Using LLMs and Classical Planning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robotic task anticipation methods rely heavily on data-driven deep networks and LLMs, which struggle with multistep, multilevel decision-making, require extensive training, and fail to leverage domain knowledge, leading to inefficient task execution and difficulty in adapting to environmental changes.
Innovation Solution
A method and system integrating large language models (LLMs) with classical planning, using a standardized prompt to predict anticipated tasks, cluster them, and convert to a PDDL format for efficient task planning, minimizing execution cost and adapting to interruptions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data-driven deep network architectures and LLMs are used for task anticipation, then task prediction capability is improved, but training time and processing power requirements increase significantly
Solution Approach 1:
The patent applies preliminary action by pre-training large language models on general task knowledge before deploying them for specific task anticipation. The LLMs are pre-trained on extensive task descriptions and workflows in advance, so that during actual deployment, they can quickly anticipate tasks without requiring extensive retraining on every new task scenario.
Solution Approach 2:
The patent introduces an intermediary classical planning system that translates LLM-generated task predictions into executable action sequences. This intermediary planner acts as a bridge between the high-level task anticipation from LLMs and the low-level robot execution, reducing the burden on the LLMs to directly handle all decision-making complexity.
2Extent of automation
If LLMs are used to generate plans with minimal human intervention, then automation level is improved, but reliability in multistep decision-making deteriorates
Solution Approach 1:
The patent segments the task planning process into distinct stages: (1) LLM generates high-level task predictions and goals, (2) classical planner breaks down goals into actionable sequences, (3) executor carries out actions, and (4) feedback loop refines predictions. This segmentation allows each component to specialize, improving overall reliability while maintaining high automation.
Solution Approach 2:
The patent implements feedback mechanisms where the robot's actual execution results and environmental changes are fed back to the LLM and planner. This allows the system to learn from past performance, correct errors in task anticipation, and adapt to unexpected situations, thereby improving reliability of multistep decision-making over time.
3Adaptability or versatility
If deep learning models are trained with demonstration videos of each task, then task understanding is improved, but device complexity and training resources increase
Solution Approach 1:
The patent applies universality by using a single pre-trained LLM that can handle multiple different task domains and types. Instead of training separate deep learning models for each specific task requiring demonstration videos, the universal LLM understands diverse task descriptions and can adapt to new tasks through prompting and few-shot learning, greatly reducing system complexity.
Solution Approach 2:
The patent substitutes the mechanical approach of collecting and processing demonstration videos for each task with a language-based approach. The LLM processes natural language task descriptions directly, replacing the need for video data collection, processing, and training pipelines with a more flexible text-based knowledge representation system.
4Ease of manufacture
If classical planning methods with PDDL are used, then domain knowledge utilization is improved, but adaptability to environmental changes deteriorates
Solution Approach 1:
The patent introduces dynamics by combining the static, rule-based PDDL planner with a dynamic LLM that can adapt to changing environments. When environmental changes occur, the LLM can update task predictions and goals based on new information, while the PDDL planner re-plans actions accordingly. This dynamic interaction allows the system to maintain formal reasoning capabilities while adapting to environmental changes.
Data Source
AI summary
The present invention generally relates to the field of robotics, and, more particularly, to a method and system for task anticipation by integrating large language models and classical planning. Conventional methods for task anticipating use data-driven deep network architectures and Large Language Models (LLMs) for task estimation but they do so at the level of high-level tasks and require a large number of training examples. Thus, embodiments of present disclosure provide a method and system for task anticipation by integrating large language models and classical planning. The disclosed method and system leverages the generic knowledge of LLMs through a small number of prompts to perform high-level task anticipation, using the anticipated tasks as joint goals in a classical planning system to compute a sequence of finer granularity actions that jointly achieve these goals.