3D Shape-Based Object Retrieval for Conveyor Sorting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automated systems for object recognition and classification on conveyor belts, such as in manufacturing and packaging, face challenges in accurately identifying and sorting diverse objects with varying sizes and characteristics, especially when objects are unsorted and intermingled, due to limitations in depth sensing and image processing technologies.
Innovation Solution
The use of three-dimensional (3D) range cameras and color cameras to capture depth and color images, generating 3D point clouds and descriptors for objects, combined with convolutional neural networks for multi-view object classification, enables robust object recognition and classification by summarizing features and invariant representations across different poses and orientations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional 2D image processing is used for object recognition, then the system complexity is low, but the classification accuracy deteriorates when objects have varying poses and orientations
Solution Approach 1:
The patent transitions from 2D image processing to 3D shape analysis by capturing depth information and generating 3D point clouds. This dimensional change enables the system to recognize objects regardless of their pose or orientation in 2D space, as the 3D shape representation is invariant to viewing angles. The 3D convolutional neural network processes this volumetric data to achieve accurate classification while maintaining robustness to pose variations.
2Measurement precision
If multiple sensors are used to capture depth and color information, then the object recognition accuracy improves, but the device complexity and cost increase
Solution Approach 1:
The patent combines depth sensing and color imaging into an integrated system that captures both types of information simultaneously. The 3D point cloud data from the depth sensor is merged with corresponding color image data, creating a rich multi-modal representation of objects. This combination allows the convolutional neural network to leverage both geometric shape information and visual appearance for more accurate classification.
3Measurement precision
If 3D point cloud processing is used for shape analysis, then the classification accuracy improves, but the computational time and processing complexity increase
Solution Approach 1:
The patent performs preliminary processing of the 3D point cloud data by generating a normalized shape representation and extracting key geometric features before feeding the data to the convolutional neural network. This preprocessing step organizes the raw point cloud into a structured format that is more efficient for neural network processing, reducing the computational burden during classification while preserving the essential shape information needed for accurate recognition.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A method for classifying physical objects includes: controlling, by a processor, one or more depth cameras to capture depth images of a query object; controlling, by the processor, one or more color cameras to capture a color images of the query object; computing, by the processor, a three-dimensional (3D) model of the query object using the depth images; combining, by the processor, the color images with the 3D model; computing, by the processor, a descriptor from the 3D model and the color images, the descriptor including: a multi-dimensional shape descriptor space representation of a 3D shape of the query object; a multi-dimensional color descriptor space representation of a texture of the query object; and a one- dimensional size descriptor space representation of a size of query object; supplying, by the processor, the descriptor to a classifier to compute a classification of the query object; and outputting the classification of the query object.