Bin Picking Vision with Coarse Pose and Stereo Refinement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine vision systems face challenges in efficiently identifying the pose of objects, especially when objects lack recognizable features or are mixed together, requiring significant computational resources and time for accurate pose determination in real-time applications like robot pick and place systems.
Innovation Solution
A data processor-connected machine vision system that uses multiple cameras or a movable camera to capture images from different viewpoints, processing 2D image data to determine a search range and perform stereo matching for accurate object pose estimation, aided by machine learning techniques such as convolutional neural networks (CNNs) for coarse pose estimation and pickability criteria evaluation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If machine vision systems process images to identify object poses in real-time, then productivity is improved, but computational resources and time consumption increase significantly
Solution Approach 1:
The patent segments the computational process into two distinct stages: a first pass that identifies candidate objects and their approximate poses, and a second pass that performs detailed pose refinement only on selected candidates. This segmentation reduces the computational burden in real-time operation by limiting expensive calculations to a subset of objects rather than processing all detected objects equally.
Solution Approach 2:
The system performs partial action by conducting comprehensive pose determination only for selected candidate objects rather than all detected objects. The first pass quickly identifies promising candidates, and the second pass applies detailed analysis only to these candidates, avoiding excessive computation on objects that will not be picked.
2Measurement precision
If machine vision systems perform detailed pose determination for all detected objects, then measurement precision is improved, but computational complexity increases
Solution Approach 1:
The patent applies local quality by providing different levels of analysis quality to different objects based on their relevance. Selected candidate objects receive detailed, high-precision pose determination, while other objects receive only preliminary assessment. This differentiated approach maintains measurement precision for critical objects while reducing overall computational complexity.
3Reliability
If robots wait for complete pose determination before picking objects, then reliability is improved, but productivity decreases
Solution Approach 1:
The system performs preliminary action by completing pose determination for selected candidate objects before the robot actually picks them. The first pass identifies candidates and their approximate poses in advance, allowing the robot to prepare and execute picks without waiting for complete analysis of all objects. This ensures reliability for picked objects while improving productivity through asynchronous processing.
Data Source
AI summary
A pick and place system comprises a computer connected to receive images of a field of view of a bin or other location at which objects are placed from disparate viewpoints. The computer is configured to process 2D image data of one or more of the images to determine a coarse pose and search range corresponding to the object. The computer is configured to perform subsequent stereo matching within the search range to obtain an accurate pose of the object. The computer is connected to control a robot to pick and place a selected object. Poses of objects may be determined asynchronously with picking the objects. Poses of plural objects may be determined and saved. the images may be processed to detect changes in the field of view. Saved poses for objects unaffected by changes may be used to pick the corresponding objects.