Hybrid 2D/3D Sensor Fusion for Automated Box Picking Adaptability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automated box picking and decanting systems face challenges such as positioning difficulties, inflexibility, and inefficiency due to reliance on classical image processing and machine learning methods, which require extensive training and are prone to overtraining or undertraining, making them costly and difficult to adapt to new items or configurations.
Innovation Solution
A method and system that utilize a hybrid 2D/3D sensor-based approach with a surprisal-based information theoretic framework to efficiently rank and select sequences of legal picks, allowing for efficient packing of transport containers by computing a weighting vector through multidimensional optimization and using a vision-guided robot to pick and place items based on ranked configurations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If classical image processing and machine learning methods are used for automated box picking, then automation capability is achieved, but the system requires extensive training and is prone to overtraining or undertraining, making it costly and difficult to adapt
Solution Approach 1:
The patent replaces complex machine learning algorithms with a geometric modeling approach using 2D/3D sensor fusion. Instead of training neural networks to recognize box configurations, the system uses analytical geometry to compute pick sequences, eliminating the need for extensive training data and making the system immediately adaptable to new items.
Solution Approach 2:
The system changes the fundamental parameters used for decision-making from learned features in machine learning to explicit geometric parameters (positions, orientations, dimensions) obtained through 2D/3D sensor fusion. This parameter transformation enables direct computation of pick sequences without training, improving adaptability while maintaining automation.
2Extent of automation
If machine learning methods are used for box configuration recognition, then automation is achieved, but the training process is time-consuming and costly
Solution Approach 1:
The patent substitutes machine learning training processes with real-time geometric computation. The system uses 2D image data combined with 3D sensor information to directly calculate box positions and orientations, eliminating the time-consuming training phase while achieving accurate configuration recognition.
Solution Approach 2:
The system performs preliminary geometric modeling and sensor calibration once during setup, then uses these pre-established models for rapid real-time recognition without requiring repeated training. This preliminary action eliminates ongoing training time losses while maintaining automation accuracy.
3Productivity
If automated systems are used for box picking, then labor is reduced, but positioning difficulties arise that humans can easily overcome
Solution Approach 1:
The patent introduces 3D sensors as an intermediary between the 2D camera and the picking system. The 3D sensors provide depth information that acts as a mediator to resolve ambiguities in 2D image positioning, enabling precise three-dimensional localization of boxes that overcomes the positioning difficulties faced by traditional automated systems.
Solution Approach 2:
The system transitions from two-dimensional image processing to three-dimensional spatial understanding by fusing 2D/3D sensor data. This dimensionality change provides accurate depth and orientation information, resolving positioning ambiguities and enabling precise box location detection that matches or exceeds human capability.
Data Source
AI summary
A method and system for efficiently packing a transport container with a plurality of substantially identical target items picked from a transport structure with a picking tool are provided. The method includes the step of providing a plurality of equivalence classes which partition all possible sequences of legal picks. Each equivalence class containing a subset of items from a ranked configuration. The method also includes ranking the equivalence classes in order of expected time efficiency under the constraint that a minimum, predetermined level of space efficiency is maintained. Finally, the method includes selecting the best ranked equivalence class for picking.