Camera-Based Robot Teaching for Intuitive Pick-and-Place Programming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for teaching industrial robots to perform pick and place operations are unintuitive, time-consuming, and costly, especially for non-expert operators, and require either a teach pendant or a costly motion capture system.
Innovation Solution
A method using a single camera to analyze images of a human hand and workpiece to determine the robot gripper pose and position, generating programming commands for the robot to replicate the human demonstration, allowing for intuitive and efficient teaching of pick, move, and place operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a teach pendant is used to program the robot, then the robot can be taught to perform pick and place operations, but the process becomes unintuitive, error-prone and time-consuming
Solution Approach 1:
The system captures and records the actual human movements and gestures performed during the demonstration, then replays these recorded movements to control the robot. This copying approach eliminates the need for manual programming while preserving the natural human operating style, making the process both intuitive and time-efficient
Solution Approach 2:
The patent replaces the mechanical teach pendant interface with an optical sensing system using cameras and image processing. This substitution captures human movements visually and translates them into robot commands, eliminating the complex manual manipulation of teach pendants while reducing programming time
2Measurement precision
If a motion capture system with multiple cameras is used, then accurate position and orientation data can be obtained, but the system becomes costly and difficult to set up
Solution Approach 1:
The system extracts only the essential visual information needed for robot teaching from the camera footage, focusing on capturing human hand and workpiece positions rather than attempting to capture complete environmental data. This extraction approach achieves sufficient measurement precision with a simplified single-camera setup
Solution Approach 2:
The single camera system is designed to perform multiple functions: capturing workpiece position, detecting human hand gestures, and providing visual feedback to the operator. This multi-functionality eliminates the need for multiple specialized cameras and complex motion capture infrastructure while maintaining adequate measurement accuracy
3Ease of manufacture
If conventional teach pendant methods are used, then robot programming can be accomplished, but the process is costly and requires expert operators
Solution Approach 1:
The system enables operators to teach the robot through natural demonstration without requiring programming expertise. The camera system automatically captures and processes the demonstration, translating human movements into robot commands autonomously. This self-service capability eliminates the need for expensive expert programmers while reducing system costs
Solution Approach 2:
The patent replaces expensive motion capture systems and complex programming interfaces with a simpler camera-based visual system. This substitution dramatically reduces equipment costs while making the teaching process accessible to non-expert operators through intuitive demonstration rather than complex programming
Data Source
AI summary
A method for teaching a robot to perform an operation based on human demonstration with images from a camera. The method includes a teaching phase where a 2D or 3D camera detects a human hand grasping and moving a workpiece, and images of the hand and workpiece are analyzed to determine a robot gripper pose and positions which equate to the pose and positions of the hand and corresponding pose and positions of the workpiece. Robot programming commands are then generated from the computed gripper pose and position relative to the workpiece pose and position. In a replay phase, the camera identifies workpiece pose and position, and the programming commands cause the robot to move the gripper to pick, move and place the workpiece as demonstrated. A teleoperation mode is also disclosed, where camera images of a human hand are used to control movement of the robot in real time.