Robot Action Library for Parallel Reinforcement Learning Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robot operation methods are inefficient as they typically execute actions sequentially, limiting the ability to perform multiple tasks in parallel and not effectively utilizing reinforcement learning for combined actions.
Innovation Solution
Generating and storing combined actions by combining original actions in an action library, then applying reinforcement learning to create a control policy that leverages both original and combined actions for improved efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple actions are executed sequentially, then the robot can complete tasks with simple control logic, but the task completion time increases and efficiency decreases
Solution Approach 1:
The patent applies dynamics by transitioning from static sequential action execution to dynamic parallel action execution. The reinforcement learning agent dynamically determines which actions can be executed in parallel based on current state, allowing the robot to adaptively optimize task completion time while maintaining control simplicity through learned policies.
Solution Approach 2:
The patent uses preliminary action by pre-defining action templates that contain multiple atomic actions. These templates are prepared in advance and can be executed as unified parallel actions, eliminating the need for real-time sequential coordination and reducing overall task completion time.
2Productivity
If combined actions are generated and stored in action library, then parallel execution improves efficiency, but the action library size and system complexity increase
Solution Approach 1:
The patent applies segmentation by decomposing complex combined actions into atomic actions that can be independently managed. The action library stores both atomic actions and pre-defined templates, allowing the system to break down complex tasks into manageable components while maintaining the ability to execute them in parallel when appropriate.
Solution Approach 2:
The patent uses parameter changes by modifying the action library structure to include templates with parameters for parallel execution. The reinforcement learning agent learns to select and parameterize templates based on task requirements, enabling efficient parallel execution without manually managing all possible action combinations.
3Loss of time
If reinforcement learning is applied to learn control policy with combined actions, then task completion time reduces, but the computational complexity and training requirements increase
Solution Approach 1:
The patent introduces templates as intermediaries between the reinforcement learning agent and atomic actions. The templates serve as pre-processed action packages that the agent can select and execute, reducing the computational burden on the agent while still enabling parallel action execution and reducing task completion time.
Solution Approach 2:
The patent applies preliminary action by pre-defining action templates with structured parameters for parallel execution. This preliminary structuring reduces the computational complexity during training and execution, as the reinforcement learning agent only needs to select and parameterize templates rather than learn all possible action combinations from scratch.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
The invention relates to a method for improving operation of at least one robot. The robot is being operated on the basis of a set of predefined actions. A method comprises: Generating combined actions (51, 53) by combining at least two actions out of a set of original actions stored in an action library. Storing the combined actions in the actions library in addition to the original actions. Applying a reinforcement learning algorithm (S4) to the set of actions stored now in the action library to learn a control policy making use of the original actions and the combined actions. And finally, operating the robot on the basis of the resulting action library.