Robotic Picking Using Surface Projection for Heap Grasp Posture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional robot systems face difficulties in accurately estimating the posture for grasping and picking up target objects, especially when objects are disorganized and in a heap, as neural networks struggle to calculate 6D grasp postures for multiple objects simultaneously.
Innovation Solution
A handling apparatus and method that utilizes a two-stage deep learning model to calculate surface information and grasp postures, involving a surface information calculation unit, projection transformation, posture calculation, and inverse projection transformation to determine optimal grasp postures for a picking tool, allowing for easier estimation of object postures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If conventional neural networks are used to calculate 6D grasp postures for multiple objects simultaneously, then automation is achieved, but measurement precision of object posture deteriorates
Solution Approach 1:
The patent divides the complex task of 6D grasp posture calculation into two separate stages: first calculating 3D position information, then calculating 3D posture information based on the position. This segmentation allows each stage to focus on specific aspects, improving overall precision while maintaining automation.
Solution Approach 2:
The patent introduces an intermediary step of calculating 3D position information as a bridge between image input and final 6D grasp posture output. This intermediate representation simplifies the learning task for the neural network and improves the accuracy of the final posture estimation.
2Device complexity
If single-stage neural networks calculate all posture information at once, then device complexity is reduced, but manufacturing precision of grasp posture deteriorates
Solution Approach 1:
The patent segments the neural network into two distinct modules: a first neural network for 3D position calculation and a second neural network for 3D posture calculation. This segmentation increases computational precision while keeping each individual network module relatively simple and manageable.
Data Source
AI summary
A handling apparatus includes a controller and a picking tool for a target object. The controller calculates surface information about a target object by using a first learning model for estimating the surface information from a feature map. The feature map represents a feature of an input image including the target object. The controller identifies surfaces in the input image from the surface information, and transforms the input image into projection images, each being seen from above a point on each of the surfaces in a normal vector direction. The controller calculates, for each projection image, a grasp posture for gripping the target object by using a second learning model. The controller transforms the grasp posture into a grasp posture in a first coordinate system. The controller calculates a final posture of the picking tool by projecting the grasp posture in the first coordinate system onto the input image.