Robot Picking Grasp Estimation Using Surface Projection Views
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional robot systems face difficulties in accurately estimating the posture for grasping and picking up target objects, especially when objects are disorganized and in a heap, as neural networks struggle to calculate 6D grasp postures for multiple objects simultaneously.
Innovation Solution
A handling apparatus and method utilizing a two-stage deep learning model to estimate surface information and grasp postures, involving a surface information calculation unit, projection transformation unit, posture calculation unit, and inverse projection transformation unit to determine a final grasp posture in a world coordinate system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a neural network is used to estimate grasp postures for multiple objects simultaneously, then the system can handle disorganized objects in a heap, but the measurement precision of 6D grasp postures deteriorates
Solution Approach 1:
The patent divides the complex task of estimating grasp postures for multiple objects into separate processing stages. First, object detection and segmentation identify individual objects in the heap. Then, for each segmented object, the system calculates surface information and estimates grasp postures independently. This segmentation approach allows the system to handle disorganized objects while maintaining measurement precision by avoiding the computational complexity of simultaneous multi-object posture estimation.
Solution Approach 2:
The patent transforms the 2D image data into 3D surface information by calculating normal vectors and surface equations for each object. This dimensional transformation enables the system to represent objects in three-dimensional space, which is crucial for accurate grasp posture estimation. By moving from 2D image processing to 3D surface representation, the system achieves both adaptability to disorganized objects and precision in posture calculation.
2Device complexity
If conventional robot systems are used for picking automation, then the system structure is simple, but the manufacturing precision of grasp posture calculation deteriorates
Solution Approach 1:
The patent performs preliminary calculations of surface information (normal vectors, surface equations) for each detected object before estimating grasp postures. This preliminary action prepares the necessary geometric data in advance, enabling more accurate grasp posture calculations without adding significant system complexity. The preliminary surface analysis is performed automatically as part of the object detection process, maintaining system simplicity while improving manufacturing precision.
3Ease of operation
If robots grasp objects from directly above, then the operation is simple, but the adaptability to disorganized and heaped objects deteriorates
Solution Approach 1:
The patent makes the grasping operation dynamic by calculating optimal grasp postures for each object based on its specific surface information and orientation. Instead of using a fixed direct-overhead grasping approach, the system adapts the grasp posture to match the actual object configuration. This dynamic adjustment of grasp parameters enables the robot to handle disorganized and heaped objects while maintaining operational simplicity through automated posture calculation.
Data Source
AI summary
A handling apparatus includes a controller and a picking tool for a target object. The controller calculates surface information about a target object by using a first learning model for estimating the surface information from a feature map. The feature map represents a feature of an input image including the target object. The controller identifies surfaces in the input image from the surface information, and transforms the input image into projection images, each being seen from above a point on each of the surfaces in a normal vector direction. The controller calculates, for each projection image, a grasp posture for gripping the target object by using a second learning model. The controller transforms the grasp posture into a grasp posture in a first coordinate system. The controller calculates a final posture of the picking tool by projecting the grasp posture in the first coordinate system onto the input image.