Hybrid 2-D/3-D Vision for Adaptive Box Picking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated systems for manipulating box-like items on a horizontal surface face challenges such as positioning difficulties, inflexibility, high implementation costs, inefficiency, and the need for extensive training when new items are introduced, particularly due to reliance on Classical Image Processing and Machine Learning methods.
Innovation Solution
A method and system utilizing a hybrid 2-D/3-D sensor to gather information about the configuration of items, generate and rank hypotheses based on depth and appearance images, and employ an autonomous manipulator to handle items efficiently, independent of pre-defined parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If Classical Image Processing and Machine Learning methods are used for automated item manipulation, then automation extent is improved, but adaptability deteriorates due to extensive retraining needed for new items
Solution Approach 1:
The patent replaces traditional Classical Image Processing and Machine Learning systems with a hybrid 2-D/3-D sensor system. This substitution eliminates the need for extensive training data and complex algorithms, allowing the system to automatically adapt to new box-like items without retraining, thereby maintaining high automation while improving adaptability.
Solution Approach 2:
The system changes the fundamental parameters used for item identification by incorporating depth information from 3-D sensors alongside 2-D images. This parameter change enables the system to distinguish and adapt to new items based on geometric and spatial characteristics rather than requiring learned patterns, thus improving adaptability while maintaining automation.
2Productivity
If automated systems are implemented for item manipulation, then productivity is improved, but device complexity increases due to positioning difficulties and inflexibility
Solution Approach 1:
The patent segments the sensing function into two distinct components: a 2-D sensor for capturing appearance and texture information, and a 3-D sensor for capturing depth and geometric information. This segmentation allows each sensor to be optimized for its specific function, simplifying the overall system architecture while maintaining high productivity through accurate item localization and recognition.
Solution Approach 2:
The hybrid sensor system acts as an intermediary between the automated manipulator and the diverse box-like items. By providing rich, multi-dimensional information about item position, orientation, and characteristics, this intermediary enables simple manipulator designs to achieve high productivity without complex positioning mechanisms.
3Ease of operation
If machine vision-based automated systems are used, then ease of operation is improved, but measurement precision deteriorates due to positioning difficulties with varied box configurations
Solution Approach 1:
The patent merges 2-D image data with 3-D depth data to create a comprehensive representation of each box-like item. This combination allows the system to accurately determine item position, orientation, and dimensions regardless of configuration variations, thereby maintaining measurement precision while preserving the ease of operation provided by automated vision-based handling.
Solution Approach 2:
The system transitions from 2-D image analysis to 3-D spatial understanding by incorporating depth information. This dimensional enhancement enables precise measurement and positioning of items in varied configurations, resolving the contradiction between ease of automated operation and measurement precision.
Data Source
AI summary
A method and system for manipulating a target item supported on a substantially horizontal support surface supporting a configuration of substantially identical items are provided. The system includes at least one light source configured to illuminate at least one surface including a top surface of the items to obtain backscattered illumination. A volumetric sensor such as a 2-D/3-D sensor is configured to sense the backscattered illumination to obtain at least one depth image and at least one appearance image. At least one processor is configured to determine geometric information and appearance information from the images. The at least one processor is also configured to generate a plurality of hypotheses and surprisals of the information and to rank the hypotheses based on their surprisals. In one embodiment, the best hypothesis is selected and the item is manipulated or picked from a pallet by a vision-guided robot based on the best hypothesis.