Skill Grounding and Goal-Conditioned Policy for Cross-Domain Agents
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional language model-based task planning technology suffers from limited scalability and generality in new domains, leading to cross-domain instruction-following problems and performance degradation in environments with reward sparsity.
Innovation Solution
A skill-based agent operation system and method that includes a skill grounding device for semantic interpretation of natural language instructions and a goal-conditioned policy learning device for constructing skill sequences through reinforcement learning, enabling rapid adaptation to new domains and robust goal achievement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If language model-based task planning technology is used to interpret user instructions and select skills, then task execution capability is improved, but scalability and generality in new domains deteriorate
Solution Approach 1:
The system segments task planning into two independent modules: skill grounding (selecting executable skills from candidate skills based on environment state) and task planning (generating action sequences from selected skills). This segmentation allows each module to be optimized independently, improving both task execution capability and adaptability to new domains.
Solution Approach 2:
The patent introduces skill grounding as an intermediary layer between instruction interpretation and task planning. This intermediary module selects appropriate executable skills based on current environment state, acting as a bridge that enables better generalization to new domains while maintaining effective task execution.
2Reliability
If Reinforcement Learning is used for goal-conditioned policy learning, then long-term goal achievement is improved, but learning efficiency and reliability deteriorate in environments with reward sparsity
Solution Approach 1:
The system implements skill grounding feedback that provides intermediate guidance signals during task execution. By selecting executable skills based on environment state, the system creates feedback loops that guide the agent toward long-term goals even when terminal rewards are sparse, improving both learning efficiency and reliability.
Solution Approach 2:
The patent performs preliminary skill selection and validation before executing task plans. By pre-selecting executable skills and validating their applicability to the current environment state, the system prepares the agent in advance, reducing exploration time and improving learning efficiency in sparse reward environments.
Data Source
AI summary
According to an embodiment of the present invention, a skill-based agent operation system may comprise a skill grounding device configured to semantically interpret a user's natural language instruction and select one or more executable candidate skills among a plurality of skills, and a goal-conditioned policy learning device configured to construct a skill sequence for achieving a goal based on the selected skill, and to generate a goal-conditioned action policy by learning the constructed skill sequence based on reinforcement learning.