Robot Picking Visual Feedback for Grasp and Drop Accuracy

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing robot systems face limitations in object picking due to limited feedback information, particularly from force feedback sensors, which hampers the accuracy and efficiency of grasping and dropping objects, especially in diverse and dynamic environments.

Innovation Solution

A feedback arrangement that utilizes imaging units to acquire images of objects after they have been picked or dropped, providing visual feedback to improve the machine learning system's control over picking and dropping actions, allowing for better determination of successful manipulations and correct placement of objects in robot systems.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If force feedback sensors are used to provide feedback information, then the robot system can detect grasping force, but the feedback information remains limited and insufficient for accurate manipulation control

Engineering Contradiction:

Improvefeedback informationVSAvoidgrasping accuracy

Core Design Contradiction:

Loss of informationVSMeasurement precision

Solution Approach 1:

The patent implements a feedback mechanism where images captured by imaging units are transmitted to a server, processed through machine learning models, and used to generate control signals that adjust robot manipulation actions in real-time, creating a closed-loop control system that continuously improves grasping accuracy

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent introduces imaging units as intermediary devices that capture visual information about objects and manipulation outcomes, serving as a bridge between the physical manipulation process and the control system, enabling richer information exchange than direct force sensors alone

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If machine learning systems are used to control picking and dropping actions, then the system can adapt to diverse objects, but the system requires extensive training data and computational resources

Engineering Contradiction:

Improveobject manipulation adaptabilityVSAvoidsystem complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent combines multiple imaging units with server-based machine learning processing to create an integrated system where visual data collection, processing, and control decision-making are merged into a unified architecture that leverages distributed computational resources

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The machine learning system automatically trains and improves through continuous processing of images from the imaging units, enabling the system to self-enhance its manipulation capabilities without requiring external retraining interventions

Inventive Principle:

Principle #25Self-service

3Measurement precision

If imaging units are added to provide visual feedback, then the accuracy of object picking and placement improves, but the system complexity and integration difficulty increase

Engineering Contradiction:

Improveobject placement accuracyVSAvoidsystem integration complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent uses imaging units as intermediary devices that capture visual information about objects and manipulation outcomes, serving as a bridge between the physical manipulation process and the control system, enabling richer information exchange without direct complex integration

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces complex mechanical feedback systems with optical imaging and digital image processing, substituting physical measurement mechanisms with visual sensing and computational analysis that can be integrated more flexibly

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Reliability

If continuous visual feedback is used for real-time control, then the robot system can correct mistakes and improve grasping success, but the data processing requirements and computational load increase

Engineering Contradiction:

Improvegrasping success rateVSAvoidcomputational energy consumption

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The patent processes images at different levels of detail and frequency, using full-resolution images only when needed for critical decisions while employing lower-resolution or sampled images for routine monitoring, reducing overall computational load while maintaining grasping success

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS11682097B2Method and an apparatus for separating at least one object from a plurality of objects

Publication Date: 2023.06.20 MP ZENROBOTICS OY

AI summary

Feedback information is an important aspect in all machine learning systems. In robot systems that are picking objects from a plurality of objects this has been arranged by acquiring images of objects that have been picked. When images are acquired after picking they can be imaged accurately and the information about picking and also dropping success can be improved by using acquired images as a feedback in the machine learning arrangement being used for controlling the picking and dropping of objects.