Robotic Pod Stowing Using Visual Perception and ML Planning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robotic systems struggle to reliably stow items into containers that may already contain various types and arrangements of items, making it difficult to manage inventory efficiently.
Innovation Solution
A robotic system equipped with sensors and machine learning techniques to analyze the environment, generate a stowing plan, and use robotic gantries and end effectors to stow items into containers while managing retaining elements and optimizing space utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If items are stowed randomly in containers with many different types and sizes, then storage capacity is maximized, but robotic control reliability deteriorates
Solution Approach 1:
The system dynamically changes parameters such as container selection, item placement position, and stowing strategy based on real-time perception of container contents. The robotic system adjusts its control parameters adaptively to match the actual arrangement of items, enabling reliable stowing in randomly filled containers by continuously updating its understanding of the container state and planning appropriate placement actions.
2Manufacturing precision
If visual sensors are used to perceive container contents, then placement precision is improved, but system complexity increases
Solution Approach 1:
The system introduces visual sensors as an intermediary between the robotic manipulator and the container contents. The sensors capture images of the container, and image processing algorithms serve as intermediaries to interpret the visual data and determine item positions. This intermediary layer enables precise placement without requiring direct mechanical sensing or complex tactile feedback systems.
Solution Approach 2:
The system replaces mechanical sensing methods with visual sensing. Instead of using complex mechanical probes or tactile sensors to detect item positions, the system uses cameras and computer vision algorithms to perceive the container contents, significantly reducing mechanical complexity while maintaining or improving measurement precision.
3Measurement precision
If multiple sensors and machine learning techniques are deployed, then stowing accuracy is improved, but computational requirements and processing time increase
Solution Approach 1:
The system performs preliminary perception and analysis of container contents before the actual stowing action. By capturing images and processing them in advance to identify available space and plan placement positions, the system prepares all necessary information beforehand, enabling fast execution of the stowing action without time-consuming computations during the critical placement phase.
Data Source
AI summary
Techniques and systems for performing a perception analysis for a robotic stowing operation are described. An example technique includes obtaining, via multiple sensors, multiple first images, wherein each first image is an image of a different container of an inventory holder within a robotic workcell. A first machine learning (ML) and image processing pipeline is performed with the first images to determine displacement locations for the containers of the inventory holder. A second ML and image processing pipeline is performed with the first images to determine content signatures for the containers. A plan is generated for stowing a first item into a first container, based at least in part on the plurality of content signatures and the plurality of displacement locations. A robotic apparatus is controlled to stow the first item into the first container, based on the plan.