Robot Teaching With Visual Servoing for Precise Workpiece Placement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for teaching industrial robots to perform pick and place operations are unintuitive, time-consuming, and lack precision, especially for non-expert operators and applications requiring precise placement, such as component installation and assembly.
Innovation Solution
A method using image-based visual servoing (IBVS) that involves a camera detecting a human hand grasping and moving a workpiece to define a rough trajectory, with line features or other geometric features collected during a demonstration phase used to refine the final placement position, enabling precise robotic movement and placement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a teach pendant is used to program the robot, then the robot can be taught to perform pick and place operations, but the process becomes unintuitive, error-prone and time-consuming
Solution Approach 1:
The system captures and records the human demonstrator's hand movements and robot operations to create a reusable teaching program. The camera records the demonstrator's hand position, orientation, and the robot's corresponding actions, which are then processed into a structured teaching program that can be executed repeatedly without requiring the operator to manually program each movement.
Solution Approach 2:
The patent replaces the manual mechanical teaching process (using teach pendant to manually move robot to positions) with an automated vision-based system. The camera captures the demonstrator's hand movements and uses image processing to automatically generate the teaching program, eliminating the need for manual mechanical positioning and programming.
2Measurement precision
If a motion capture system is used to teach the robot, then positional accuracy can be improved, but the system becomes costly and difficult to set up
Solution Approach 1:
The patent introduces a camera as an intermediary device between the human demonstrator and the robot teaching process. Instead of using complex motion capture systems with multiple cameras and markers, a single camera captures the demonstrator's hand movements and the workpiece position, which are then processed to generate the teaching program with sufficient accuracy for the application.
3Ease of operation
If human demonstration is used to teach the robot, then the teaching process becomes simpler and more intuitive, but the positional accuracy is insufficient for precise placement operations
Solution Approach 1:
The system incorporates feedback by continuously monitoring the demonstrator's hand position and the robot's actual movements during the demonstration phase. The recorded data is processed to identify the relationship between hand position and robot position, and this feedback is used to automatically generate the teaching program with corrected positional information, ensuring both simplicity and precision.
Data Source
AI summary
A method for teaching and controlling a robot to perform an operation based on human demonstration with images from a camera. The method includes a demonstration phase where a camera detects a human hand grasping and moving a workpiece to define a rough trajectory of the robotic movement of the workpiece. Line features or other geometric features on the workpiece collected during the demonstration phase are used in an image-based visual servoing (IBVS) approach which refines a final placement position of the workpiece, where the IBVS control takes over the workpiece placement during the final approach by the robot. Moving object detection is used for automatically localizing both object and hand position in 2D image space, and then identifying line features on the workpiece by removing line features belonging to the hand using hand keypoint detection.