3D Shelf Obstruction Detection for Mixed-Depth Object Positioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The varying structure of shelves in environments such as retail facilities complicates object detection, reducing the accuracy of status information detected from captured data, particularly when shelf edges have mixed depths, affecting the precision of identifying objects and obstructions.
Innovation Solution
A method involving a mobile automation apparatus equipped with image and depth sensors that generates three-dimensional positions of objects by projecting two-dimensional boundaries into point clouds, and detects obstructions directly from point clouds using local support structure planes, employing cost functions to refine object positions and identify obstructions robustly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional object detection methods are used on captured data, then the detection process is simple, but the accuracy of status information is reduced due to varying shelf structures and mixed depths
Solution Approach 1:
The patent transforms two-dimensional image data into three-dimensional point cloud data, adding a depth dimension to the detection process. This dimensional transformation enables accurate representation of objects on shelves with varying depths and mixed structures, resolving the accuracy issue while maintaining manageable system complexity through software-based processing.
Solution Approach 2:
The patent changes the detection parameters by introducing depth information and three-dimensional spatial coordinates. By transforming from 2D image coordinates to 3D point cloud coordinates with depth values, the system can accurately detect objects on shelves with mixed depths, significantly improving measurement precision without requiring complex hardware modifications.
2Measurement precision
If 3D position generation from 2D boundaries is implemented, then object position accuracy is improved, but the processing complexity increases
Solution Approach 1:
The patent introduces an intermediary coordinate transformation process that maps 2D image boundaries to 3D point cloud positions through depth information. This intermediary step uses the known camera parameters and point cloud depth data to calculate accurate 3D positions, improving precision while keeping processing complexity manageable through systematic mathematical transformations.
Solution Approach 2:
The patent performs preliminary actions by first capturing both 2D images and 3D point cloud data simultaneously, then pre-processing the point cloud to identify shelf structures and depth information before generating object positions. This preliminary preparation of spatial data enables accurate 3D position generation without requiring complex real-time processing during object detection.
3Reliability
If detection accuracy is improved through 3D point cloud analysis, then the reliability of product status monitoring is enhanced, but the computational requirements and processing time increase
Solution Approach 1:
The patent segments the detection process into distinct stages: point cloud acquisition, shelf structure identification, object boundary detection, and position calculation. By dividing the complex 3D detection task into manageable segments, the system achieves high reliability in product status monitoring while reducing overall processing time through parallel processing of different detection stages.
Solution Approach 2:
The patent applies partial action by focusing detection efforts on relevant regions identified in the 2D image boundaries, rather than processing the entire point cloud. By limiting 3D position generation to only those areas where objects are detected in 2D, the system maintains high reliability for monitored products while significantly reducing unnecessary computational overhead and processing time.
Data Source
AI summary
A method includes obtaining (i) a point cloud, captured by a depth sensor, of a structure and an obstruction, and (ii) a plurality of local structure planes derived from the point cloud and corresponding to respective portions of the structure, for each local structure plane: selecting a membership set of points from the point cloud, generating a mask based on the membership set of points, selecting a subset of points from the point cloud based on the local structure plane and the mask, and detecting obstructions from the subset of points.