Robot End Effector Placement Using Vision-Guided ML Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Robots programmed to place objects in fixed locations fail in dynamic or varying environments without explicit reprogramming, as they cannot adapt to changes in the location of objects or environments.
Innovation Solution
A machine learning model, such as a deep neural network, is trained to process images from a robot's vision component, evaluate candidate end effector actions, and predict the likelihood of successful placement of an object in a target location, allowing continuous adjustment and adaptation to new environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a robot is programmed to place objects in preprogrammed fixed locations, then the robot can perform placement tasks reliably in static environments, but the robot fails to adapt when the environment changes or objects are moved to different locations
Solution Approach 1:
The system transitions from static preprogrammed location-based control to dynamic vision-guided control. The robot continuously captures images of the environment and object positions, processes these images to determine current locations, and dynamically adjusts placement targets based on real-time environmental conditions rather than relying on fixed preprogrammed coordinates
Solution Approach 2:
The system implements a feedback loop where the robot captures images of the environment and object positions, processes these images to determine current locations, compares them with target locations, and adjusts its placement actions accordingly. This closed-loop control enables the robot to adapt to environmental changes and maintain high placement success rates in dynamic settings
2Adaptability or versatility
If explicit user programming is used to reprogram the robot for new environments, then the robot can adapt to specific new locations, but this requires significant time and user intervention
Solution Approach 1:
The robot performs self-programming by autonomously capturing images of the new environment, processing these images to identify object positions and target locations, and automatically generating updated placement parameters without requiring user intervention. The vision processing system enables the robot to learn and adapt to new environments independently, eliminating the need for time-consuming manual reprogramming
Data Source
AI summary
Training and/or use of a machine learning model for placement of an object secured by an end effector of a robot. A trained machine learning model can be used to process: (1) a current image, captured by a vision component of a robot, that captures an end effector securing an object; (2) a candidate end effector action that defines a candidate motion of the end effector; and (3) a target placement input that indicates a target placement location for the object. Based on the processing, a prediction can be generated that indicates likelihood of successful placement of the object in the target placement location with application of the motion defined by the candidate end effector action. At many iterations, the candidate end effector action with the highest probability is selected and control commands provided to cause the end effector to move in conformance with the corresponding end effector action. When at least one release criteria is satisfied, control commands can be provided to cause the end effector to release the object, thereby leading to the object being placed in the target placement location.