Robot State Transition Model Aggregation for Fast Control Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for robots to acquire control rules for new work operations are inefficient, requiring long training times and struggling with adaptation due to model errors and exclusive design for specific operations.
Innovation Solution
A learning device that creates an aggregate state transition model by combining multiple state transition models, aggregating their predictions, and updating the model to minimize errors, allowing for rapid adaptation and command generation that maximizes rewards.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If model-free reinforcement learning is used to learn control rules, then the robot can autonomously acquire control rules, but the training time becomes excessively long
Solution Approach 1:
The patent applies preliminary action by pre-training state transition models on multiple source tasks before deploying them to the target task. These pre-learned models serve as initial knowledge that can be quickly adapted to new tasks, eliminating the need for lengthy training from scratch. The aggregate state transition model combines predictions from multiple pre-trained models to provide accurate initial state predictions for the target task.
Solution Approach 2:
The patent uses copying by creating multiple state transition models from different source tasks and aggregating their predictions. Instead of learning one model from scratch for each target task, the system copies knowledge from multiple source domains and combines them to form an aggregate model that can be quickly adapted to new tasks with minimal training time.
2Adaptability or versatility
If transfer learning is applied to adapt models to new work operations, then the robot can reuse learned knowledge, but the learning time becomes excessively long and contact-based operations cannot be effectively transferred
Solution Approach 1:
The patent applies segmentation by dividing the complex transfer learning problem into multiple independent state transition models, each trained on a specific source task. Instead of attempting to transfer one monolithic model across all tasks, the system segments knowledge into task-specific models and aggregates their predictions, enabling efficient adaptation to new tasks while avoiding the pitfalls of general transfer learning.
Solution Approach 2:
The patent achieves universality by creating an aggregate state transition model that can handle multiple different tasks through combining predictions from multiple source-task-specific models. This aggregate model serves as a universal foundation that can be adapted to various target tasks including contact-based operations, providing both versatility and efficiency.
3Manufacturing precision
If state transition models are trained on specific work operations, then accurate control rules are obtained, but the models cannot be effectively applied to new or similar work operations
Solution Approach 1:
The patent applies merging by combining multiple task-specific state transition models into an aggregate model. Each individual model maintains high accuracy for its specific source task, while their aggregated predictions provide broad applicability to new tasks. The aggregation process merges knowledge from multiple sources to create a model that both preserves task-specific accuracy and achieves cross-task versatility.
Solution Approach 2:
The aggregate state transition model serves as an intermediary between task-specific models and target tasks. It mediates the transfer of knowledge by aggregating predictions from multiple source-task models and providing adapted predictions for new target tasks, enabling accurate control rule application across different work operations without direct task-specific training.
Data Source
AI summary
This learning device comprises: a creation unit which creates a state transition model that predicts a next state of a robot on the basis of a measured robot state and a command for the robot, and a collection state transition model including a collection unit that collects the prediction results; a command generation unit which executes, for each control period, processes for inputting the measured robot state, generating candidates of the command for the robot, acquiring a robot state predicted from the robot state and the candidates of the command for the robot by using the collection state transition model 20, and generating and outputting a command for maximizing a reward corresponding to the acquired state; and a learning unit which updates the collection state transition model in order to reduce an error between a next robot state predicted in correspondence with the output command and a robot state measured in correspondence with the next state.