Robotic Bin Pose Detection Using Depth-Image Edge Fitting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current robotic bin-picking systems face inefficiencies in estimating the pose of bins due to technical challenges such as dynamic environments, noisy sensor data, and lack of accurate system information, leading to errors in grasp computations and reduced performance.
Innovation Solution
An autonomous system equipped with a depth camera and processor that generates a segmentation mask of the bin, scans it to identify points on the outermost edges, and fits models to determine the bin's pose, allowing for precise bin pose estimation without the need for artificial markers or detailed CAD models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current approaches are used to estimate bin pose, then the system can operate with simple sensors, but the pose estimation accuracy is insufficient leading to errors in grasp computations
Solution Approach 1:
The patent segments the bin detection process into multiple stages: generating a segmentation mask from depth images, scanning the mask to identify outermost edge points, and fitting geometric models to these points. This segmentation allows each stage to be optimized independently, improving overall pose estimation accuracy while maintaining computational efficiency for reliable grasp computations.
Solution Approach 2:
The patent transitions from 2D depth image data to 3D pose estimation by scanning the segmentation mask to identify points in multiple directions and fitting 3D geometric models. This dimensional transformation enables accurate pose determination from 2D sensor data, resolving the contradiction between using simple sensors and achieving high measurement precision.
2Measurement precision
If the system uses detailed CAD models or artificial markers to improve bin pose estimation, then measurement precision improves, but device complexity and ease of operation deteriorate
Solution Approach 1:
The system performs self-service by automatically generating segmentation masks from depth images and autonomously identifying bin boundaries and poses without requiring external markers or pre-loaded CAD models. The algorithm adapts to different bin types and positions automatically, maintaining high measurement precision while minimizing device complexity and improving ease of operation.
Solution Approach 2:
The patent changes the approach from using fixed CAD models or artificial markers to dynamically generating segmentation masks and adapting the detection parameters based on the actual bin geometry observed in the depth image. This parameter adaptation allows accurate pose estimation for various bin types without increasing system complexity.
3Ease of operation
If the system processes noisy sensor data from low-cost depth cameras, then ease of operation and cost-effectiveness improve, but measurement precision and reliability worsen
Solution Approach 1:
The patent converts the harmful effect of noisy sensor data into a benefit by using the segmentation mask scanning approach to identify outermost edge points, which are more reliable features even in noisy data. The geometric model fitting process further filters out noise by finding the best-fit parameters, thereby achieving high measurement precision while maintaining ease of operation with low-cost sensors.
Solution Approach 2:
The segmentation mask serves as an intermediary between the raw noisy depth image and the final pose estimation. By processing the depth data through segmentation and edge point identification, the system creates a cleaner intermediate representation that improves measurement precision while maintaining compatibility with low-cost sensors and ease of operation.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
It is recognized herein that current approaches to robotic picking lack efficiency and capabilities. In particular, current approaches often do not properly or efficiently estimate the pose of bins, due to various technical challenges in doing so, which can impact grasp computations and overall performance of a given robot. The pose of the bin can be determined or estimated based on depth images. Such bin pose estimation can be performed during runtime of a given robot, such that grasping can be enhanced due to the bin pose estimations.