Robotic Bin Picking with Perturbation and 3D Vision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Robotic devices face challenges in picking items from bins due to physical constraints, varied item positions, and materials, leading to frequent human intervention and failure in accessing all items, especially those in difficult spots or with awkward shapes.
Innovation Solution
A robotic device equipped with a picking element, such as a suction-based mechanism, and a perturbation element, like a compressed air valve, which rearranges items within the bin to facilitate successful picking by generating three-dimensional information from two-dimensional images using computer-vision systems and deep neural networks for strategic decision-making.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If a robotic device uses a picking element to retrieve items from a bin, then item retrieval can be automated, but the device fails to access items in difficult spots or with awkward shapes due to physical constraints
Solution Approach 1:
The system segments the item retrieval task into two distinct operations: picking (using the picking element) and perturbation (using the perturbation element). This segmentation allows the device to handle different aspects of the retrieval challenge separately, using perturbation to prepare items for subsequent picking attempts.
Solution Approach 2:
The perturbation element performs preliminary actions by rearranging items in the bin before picking attempts are made. This preliminary rearrangement brings difficult-to-reach items into more accessible positions, increasing the likelihood of successful picking without requiring the picking element to directly access challenging locations.
2Extent of automation
If the robotic device attempts to pick all items directly, then automation is maintained, but frequent human intervention is required due to failed picking attempts
Solution Approach 1:
The system implements self-service by autonomously performing perturbation operations when picking failures are detected. The device monitors its own performance, identifies failed picking attempts, and automatically executes perturbation sequences to correct the situation without requiring human intervention.
Solution Approach 2:
The system uses feedback from picking attempt outcomes to trigger perturbation operations. When the computer-vision system detects that a picking attempt failed or that items are in difficult-to-access positions, this feedback initiates automated perturbation sequences, creating a closed-loop control system that continuously improves retrieval success.
3Device complexity
If the robotic device uses a simple picking mechanism, then device complexity is reduced, but the ability to handle various shapes and materials is limited
Solution Approach 1:
The perturbation element acts as an intermediary that prepares items for the simple picking mechanism. By rearranging items into favorable positions and orientations, the perturbation element enables the simple picking mechanism to successfully handle diverse shapes and materials without requiring the picker itself to be complex or adaptable.
Solution Approach 2:
The system replaces complex mechanical adaptability in the picking mechanism with a combination of computer-vision-based detection and perturbation-based preparation. Instead of making the picking element mechanically adaptable to various shapes, the system uses vision to identify item characteristics and perturbation to position items suitably for the simple picker.
4Reliability
If the robotic device increases picking attempts, then item retrieval probability improves, but the need for human intervention increases due to repeated failures
Solution Approach 1:
The system dynamically adjusts its strategy by switching between picking and perturbation modes based on real-time conditions. When picking success probability is low (detected through computer vision or failed attempts), the system dynamically introduces perturbation operations to change the state of items in the bin, thereby improving subsequent picking probability without simply repeating failed picking attempts.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The robotic device effectively increases the probability of picking items by rearranging them using perturbation strategies, reducing the need for human intervention and improving the ability to handle various shapes and materials, thereby enhancing the efficiency and reliability of item retrieval.
Implementation Method 1
A robotic device may include a picking element and a perturbation element coupled to the robotic arm. The picking element may be used to pick up items in the bin.
Implementation Method 2
The perturbation element may be used to blow high-pressure air into the bin in order to perturb the contents of the bin.
Data Source
AI summary
Various embodiments of the present technology generally relate to robotic devices, computer-vision systems, and artificial intelligence. More specifically, some embodiments relate to a computer-vision system for robotic devices. In some implementations, a computer-vision is coupled to a robotic arm for picking items from a bin and perturbing items in a bin. A computer-vision system, in accordance with the present technology, may use at least two two-dimensional (2D) images to generate three-dimensional (3D) information about the bin and items in the bin. A method of training artificial neural networks to generate 3D information from 2D images includes obtaining ground truth data for a scene, obtaining at least two images of the scene, and providing the ground truth data and the at least two images to the artificial neural network configured to generate a depth map from the images based on the ground truth data.