3D Item Recognition Using Point Clouds for Stacked Object Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing item recognition systems rely solely on 2D data, leading to inefficiencies and inaccuracies in segmentation and identification, particularly when items are stacked or oddly shaped.
Innovation Solution
A method and system that utilizes 3D visual data, including point clouds and height maps, to generate region masks and surface reconstructions for improved item segmentation and identification, leveraging 3D geometric segments and 2D image segments for accurate classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If 2D data is used for item recognition, then the system is simpler to implement, but segmentation and identification accuracy deteriorates especially for stacked or oddly shaped items
Solution Approach 1:
The patent transitions from 2D image data to 3D point cloud data by introducing depth information through time-of-flight sensors. This dimensional change enables accurate representation of stacked and oddly shaped items, resolving the accuracy problem while maintaining reasonable system complexity through integrated sensor arrays.
Solution Approach 2:
The patent applies segmentation by dividing the 3D point cloud into distinct item regions using height maps and region masks. This segmentation approach accurately separates stacked items and handles complex geometries, improving identification accuracy without requiring overly complex processing algorithms.
2Measurement precision
If 3D visual data is used for item recognition, then segmentation and identification accuracy is improved, but processing complexity and time increases
Solution Approach 1:
The patent performs preliminary processing by generating height maps and region masks from the 3D point cloud before final item identification. This preliminary action simplifies the subsequent classification process by pre-segmenting the data, thereby reducing overall processing complexity while maintaining high accuracy.
Solution Approach 2:
The patent extracts key geometric features from the 3D point cloud by generating height maps and convex hulls. This extraction process isolates the essential shape information needed for identification, reducing processing complexity by eliminating unnecessary data while preserving accuracy.
3Measurement precision
If 3D visual data is used for item recognition, then item segmentation and identification accuracy is improved, but processing time increases
Solution Approach 1:
The patent replaces traditional sequential processing with parallel processing of multiple 3D sensors and image streams. This substitution enables simultaneous capture and processing of depth and color data, maintaining high accuracy while significantly reducing processing time through concurrent operations.
Solution Approach 2:
The patent performs preliminary 3D reconstruction and height map generation in parallel with image capture. This preliminary action prepares the data structure in advance, enabling faster subsequent processing and classification without compromising identification accuracy.
Data Source
AI summary
In variants, a method for item recognition can include: optionally calibrating a sampling system, determining visual data using the sampling system, determining a point cloud, determining region masks based on the point cloud, generating a surface reconstruction for each item, generating image segments for each item based on the surface reconstruction, and determining a class identifier for each item using the respective image segments.