Robot Skill Learning Using Inverse RL for Precise Force Placement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for teaching industrial robots to perform precise workpiece placement operations, such as component installation, are unintuitive, time-consuming, and prone to errors, especially when using force controllers, which require accurate positional adjustments and differ significantly from human demonstrations.
Innovation Solution
The method combines inverse reinforcement learning to capture human skills during a demonstration phase and reinforcement learning to train a robot controller using a reward function that includes a Kullback-Leibler divergence calculation, enabling the robot to mimic human precision and adapt to force-controlled environments by optimizing its actions based on probability distributions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If human demonstration data is used directly to control robot motion, then the robot can mimic human skills, but the positional accuracy is insufficient for precise placement operations
Solution Approach 1:
The patent introduces force control signals as an intermediary between human demonstration and robot motion execution. Instead of directly mapping human motion data to robot positions, the system captures force signals during human demonstration and uses them to control robot motion through force control mode, enabling both intuitive teaching and precise placement
Solution Approach 2:
The patent replaces direct mechanical motion copying with a force-based control system. Rather than mechanically replicating human motion trajectories, the system substitutes mechanical motion control with force control, where force signals captured during demonstration are used to drive robot motion, achieving higher precision through force feedback
2Manufacturing precision
If conventional teach pendant methods are used to program robot motion, then positional accuracy can be achieved, but the programming process becomes time-consuming and error-prone
Solution Approach 1:
The patent performs preliminary capture of force control signals during a single human demonstration, storing these signals for subsequent robot execution. This preliminary action eliminates the need for time-consuming incremental programming while preserving the ability to achieve precise placement through the pre-captured force signals
3Manufacturing precision
If motion capture systems are used to teach robot operations, then positional accuracy improves, but the system cost and set-up time increase significantly
Solution Approach 1:
The patent copies force control signals from human demonstration instead of using complex motion capture systems. By capturing and replaying force signals through a simplified system, the patent achieves accurate robot teaching without requiring expensive motion capture infrastructure
Solution Approach 2:
The patent enables the robot to learn and execute tasks through self-service using pre-captured force signals. The system stores force signals during demonstration and automatically retrieves and applies them during robot execution, eliminating the need for complex real-time motion capture and processing systems
Data Source
AI summary
A method for teaching a robot to perform an operation including human demonstration using inverse reinforcement learning and a reinforcement learning reward function. A demonstrator performs an operation with contact force and workpiece motion data recorded. The demonstration data is used to train an encoder neural network which captures the human skill, defining a Gaussian distribution of probabilities for a set of states and actions. Encoder and decoder neural networks are then used in live robotic operations, where the decoder is used by a robot controller to compute actions based on force and motion state data from the robot. After each operation, the reward function is computed, with a Kullback-Leibler divergence term which rewards a small difference between human demonstration and robot operation probability curves, and a completion term which rewards a successful operation by the robot. The decoder is trained using reinforcement learning to maximize the reward function.