Grasp Pose Detection Using RGBD Segmentation in Cluttered Bins
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current grasp pose detection systems face challenges in accurately identifying graspable regions due to factors like partial occlusion, poor illumination, shape and size changes of deformable objects, and restricted field of view, especially in cluttered environments, making it difficult for robots to manipulate objects effectively.
Innovation Solution
A processor-implemented method using a neural network to detect target objects from RGB images, combined with depth data and region growing techniques, and a Gaussian Mixture Model for segmenting surfaces, determines optimal fit shape primitives to identify grasp poses, overcoming limitations by using color and curvature features without requiring accurate 3D geometry or object identity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If methods rely on accurate geometric information about the object (CAD model), then manufacturing precision is improved, but adaptability deteriorates making them impractical in real-world use cases
Solution Approach 1:
The patent uses RGBD point cloud data as a digital copy of the object's surface geometry, replacing the need for precise CAD models. This copy is sufficient for grasp detection without requiring exact manufacturing precision, thereby maintaining adaptability while achieving functional accuracy.
Solution Approach 2:
The patent transforms the problem from requiring precise geometric parameters (CAD models) to using statistical surface properties (point cloud distributions, curvature, normals). This parameter transformation enables real-world applicability while maintaining sufficient precision for grasp detection.
2Adaptability or versatility
If methods compute grasp pose directly from RGBD point cloud without object identity, then adaptability is improved, but measurement precision deteriorates due to partial occlusion and poor illumination
Solution Approach 1:
The patent segments the RGBD point cloud into object regions and background regions using surface normal analysis and clustering. This segmentation isolates the object of interest from occlusions and environmental noise, improving measurement precision while maintaining adaptability to unknown objects.
Solution Approach 2:
The patent introduces surface normal vectors and curvature calculations as intermediary features between the raw point cloud and grasp pose detection. These intermediaries provide robust geometric descriptors that are invariant to illumination and partial occlusion, improving precision without sacrificing adaptability.
3Measurement precision
If region growing technique is applied on depth data, then surface determination accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent applies region growing with locally adapted criteria based on surface normal consistency and curvature continuity. By using local geometric properties to guide the region growing process, the patent achieves high surface detection accuracy while avoiding excessive computational complexity through localized rather than global processing.
4Measurement precision
If Gaussian Mixture Model is used for surface segmentation, then object segment separation accuracy is improved, but use of energy increases
Solution Approach 1:
The patent applies Gaussian Mixture Model segmentation only to the segmented object regions rather than the entire point cloud. This partial application maintains high segmentation accuracy while significantly reducing computational energy consumption by limiting the complex processing to necessary regions only.
Data Source
AI summary
Systems and methods for detecting grasping poses for handling target objects is disclosed. The system solves problem of grasp pose detection and finding suitable graspable affordance for picking objects from a confined and cluttered space, such as the bins of a rack in a retail warehouse by creating multiple surface segments within bounding box obtained from a neural network based object recognition module. Surface patches are created using a region growing technique in depth space based on surface normals directions. A Gaussian Mixture Model based on color and depth curvature is used to segment surfaces belonging to target object from background, thereby overcoming inaccuracy of object recognition module trained on a smaller dataset resulting in larger bounding boxes for target objects. Target object shape is identified by using empirical rules on surface attributes thereby detecting graspable affordances and poses thus avoiding collision with neighboring objects and grasping objects more successfully.