3D Camera Pallet Localization Using Point Cloud Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional technologies for pallet localization in warehouses are imprecise, especially when the true orientation of the pallet is unknown, and require significant computational effort, necessitating a need for more accurate and efficient detection methods.
Innovation Solution
The use of a 3D camera attached to a fork lift to collect point cloud data, which is then projected onto a 2D plane and processed using cross-correlation with known pallet templates to determine the precise location and orientation of pallet pockets, allowing for accurate alignment of the forks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional 2D laser scanners or 2D cameras are used for pallet localization, then the system complexity is reduced, but the measurement precision deteriorates when the true orientation of the pallet is unknown
Solution Approach 1:
The patent transitions from 2D laser scanners and 2D cameras to a 3D time-of-flight camera that captures point cloud data with depth information. This dimensional upgrade allows the system to determine pallet orientation and localization in three-dimensional space, resolving the precision limitation when pallet orientation is unknown while maintaining manageable system complexity through software-based point cloud processing
2Ease of operation
If image processing algorithms with template matching are used, then the ease of operation is improved, but the productivity deteriorates due to significant computational effort
Solution Approach 1:
The patent pre-processes the point cloud data by projecting it onto a 2D plane and applying distance transforms before performing template matching. This preliminary action creates a distance map that enables faster correlation calculations during the actual detection process, reducing the computational burden while maintaining the ease of template-based detection
3Reliability
If bump sensors are used to verify fork insertion, then the device complexity is reduced, but the reliability deteriorates when pallet orientation is not known accurately
Solution Approach 1:
The patent replaces the mechanical bump sensor verification system with an optical sensing approach using the 3D time-of-flight camera. The camera continuously monitors the spatial relationship between the forks and pallet pockets, providing reliable verification of correct insertion based on visual feedback rather than mechanical contact, thereby improving reliability without significantly increasing system complexity
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This method provides improved accuracy, safety, and cost-effectiveness in pallet detection, even with partially obstructed or damaged pallets, and can be applied to both manned and unmanned fork lifts, enhancing the precision and efficiency of pallet pick operations.
Implementation Method 1
a 3D camera (e.g., a time of flight camera) attached to a fork lift is used to collect point cloud data
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods for localizing pallets using images based on point cloud data are disclosed. In one embodiment, a method for localizing a pallet includes acquiring, by a 3D camera, a first image of point cloud data, the first image being representative of the pallet. The method also includes generating a second image by (1) truncating the point cloud data of the first image, and (2) orthogonally projecting the remaining point cloud data of the first image. The method further includes generating a third image by creating a binary image from the second image, and generating a cross-correlation image by cross-correlating the third image with a template of a pallet pocket. The method also includes determining a rotation angle (R) of the pallet by analyzing the cross-correlation image.