Retail Product Facing Detection with Depth–Image Edge Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems struggle to accurately detect individual products and their status in retail and distribution environments due to variability in products and data capture conditions such as lighting, leading to inaccurate product facing detection.
Innovation Solution
A method and system utilizing depth and image sensors to generate candidate facing edges, combine these edges to form boundaries, and validate them using region of interest indicators for precise product facing detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If single sensor (image or depth) is used for product detection, then device complexity is reduced, but measurement precision deteriorates due to variability in lighting and product appearance
Solution Approach 1:
The patent combines depth sensor data and image sensor data into a unified detection framework. The depth sensor provides three-dimensional spatial information while the image sensor provides visual appearance data. By merging these complementary data sources, the system achieves accurate product facing detection that overcomes the limitations of using either sensor type alone, particularly in variable lighting conditions.
Solution Approach 2:
The patent introduces a point cloud as an intermediary representation that bridges depth sensor data and image sensor data. The point cloud serves as a common framework where depth information and image information can be integrated and processed together, enabling the system to leverage both data types for improved detection accuracy without directly complicating the sensor hardware architecture.
2Measurement precision
If multiple sensors are used to improve detection accuracy, then measurement precision improves, but data processing complexity increases
Solution Approach 1:
The patent segments the detection process into distinct stages: generating candidate facing boundaries from depth data, generating candidate facing boundaries from image data, and then combining these candidates. This segmentation allows each sensor type to be processed independently through specialized algorithms, reducing the overall processing complexity compared to attempting to process all sensor data simultaneously as a unified complex task.
Solution Approach 2:
The patent creates simplified representations (point clouds and candidate boundary sets) that copy the essential features of the complex sensor data in a processed form. These simplified copies can be easily combined and processed, reducing the complexity of handling the original raw sensor data while preserving the information needed for accurate detection.
3Reliability
If traditional image processing alone is used, then device complexity is low, but reliability deteriorates under varying lighting conditions
Solution Approach 1:
The patent changes the fundamental parameters used for detection by incorporating depth information (spatial parameters) alongside image information (visual parameters). This parameter change allows the system to detect product facings based on three-dimensional geometry rather than relying solely on two-dimensional visual appearance, making the detection reliable under varying lighting conditions where visual parameters may be unreliable.
Data Source
AI summary
A method of detecting product facings from captured depth and image data includes: obtaining, at an imaging controller, (i) depth measurements representing a support structure supporting a plurality of product facings, (ii) image data representing the support structure, and (iii) a set of region of interest (ROI) indicators each indicating a position of a plurality of the product facings; generating a first set of candidate facing edges from the depth measurements; generating a second set of candidate facing edges from the image data; generating a third set of candidate facing edges by combining the first and second sets; generating, for each adjacent pair of the third set of candidate facing edges, a candidate facing boundary; selecting a subset of output facing boundaries from the candidate facing boundaries, based on the ROI indicators; and storing the output facing boundaries in a memory coupled to the imaging controller.