Skill Discovery for Imitation Learning with Noisy Data

The patent segments the training demonstrations into two distinct categories: high-quality demonstrations and noisy demonstrations. This segmentation allows the system to process and learn from different types of data separately, extracting useful information from noisy demonstrations while maintaining the quality standards set by high-quality demonstrations. The segmentation resolves the contradiction by enabling the model to leverage both small amounts of high-quality data and large amounts of noisy data without compromising training accuracy.

The patent introduces skill discovery as an intermediary process that bridges high-quality and noisy demonstrations. By first discovering skills from high-quality demonstrations and then using these skills to guide learning from noisy demonstrations, the system acts as a mediator that filters and processes information. This intermediary approach allows the model to benefit from the quantity of noisy demonstrations while maintaining the quality standards established by the smaller set of high-quality demonstrations.

The patent converts the harmful effect of noisy demonstrations into a benefit by using skill discovery to extract useful patterns and skills from the noise. Instead of discarding noisy demonstrations due to their low quality, the system processes them through a skill-based framework that identifies and learns from valid action sequences embedded within the noise. This transforms the previously harmful noisy data into a valuable training resource, increasing data quantity without sacrificing accuracy.

The skill matching model serves as an intermediary that filters noisy demonstrations before they reach the main training process. By matching demonstrations against discovered skills, the system identifies and utilizes only the useful portions of noisy demonstrations while filtering out the noise. This intermediary filtering mechanism enables the model to leverage large quantities of noisy data while maintaining training accuracy through selective learning.

The patent segments the training process into distinct phases: skill discovery from high-quality demonstrations, skill matching with noisy demonstrations, and end-to-end training. This segmentation provides clear interpretability for each stage while managing complexity through structured progression. Each segment has a specific purpose and can be analyzed independently, maintaining interpretability even as the overall process complexity increases.

The patent maintains continuity of useful action by ensuring that skills discovered in the first phase continue to be utilized and refined in subsequent phases. The skill matching model continuously refines the discovered skills using noisy demonstrations, and the end-to-end training further optimizes them. This continuous refinement process maintains interpretability by building upon previously established skills rather than restarting the discovery process, thereby managing complexity through iterative improvement.