On-Shelf Commodity Detection Using Depth and Regular Camera Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing on-shelf merchandise detection systems in shopping scenarios face challenges in rapidly and accurately detecting changes in merchandise without consuming excessive system resources, particularly in determining the quantity and type of items picked from shelves.
Innovation Solution
The system employs at least one depth camera above the shelf to capture depth images and regular cameras on each tier to capture images of merchandise, detecting hand motion and associating each pixel with a tier, allowing for the comparison of initial and final images to determine changes in merchandise quantity and type, with optional wide-angle or fisheye cameras for comprehensive coverage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple cameras are deployed to capture comprehensive shelf images, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The shelf is divided into multiple tiers, with dedicated cameras assigned to specific tiers. This segmentation allows each camera to focus on a specific region, improving detection precision while managing system complexity through modular deployment
Solution Approach 2:
The system incorporates both depth cameras and regular cameras to capture images from different dimensional perspectives. Depth cameras provide 3D spatial information while regular cameras provide detailed 2D images, creating a multi-dimensional detection system that improves accuracy without requiring a single complex camera
2Productivity
If continuous image processing is performed to detect merchandise changes, then productivity is improved, but use of energy increases
Solution Approach 1:
Instead of continuous processing, the system performs image processing periodically or event-driven. Images are captured continuously but only processed when change detection is triggered, reducing energy consumption while maintaining detection productivity
Solution Approach 2:
The system performs preliminary comparison of depth images to detect hand motions before processing detailed merchandise images. This preliminary action filters out unnecessary full-image processing, reducing energy consumption while maintaining fast detection response
3Measurement precision
If detailed image comparison is performed to identify merchandise changes, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The system extracts only the changed regions between images for detailed analysis, rather than processing entire images. This extraction of difference regions maintains detection precision while significantly reducing processing time
Solution Approach 2:
The system performs preliminary comparison of depth images to identify regions of interest before processing detailed merchandise images. This preliminary action narrows down the processing scope, reducing time loss while maintaining accurate change detection
4Reliability
If comprehensive merchandise monitoring is implemented across all shelves, then reliability is improved, but device complexity increases
Solution Approach 1:
The monitoring system is segmented into multiple independent camera units, each responsible for specific shelf tiers. This segmentation improves reliability by isolating failures to individual segments while managing complexity through standardized modular units
Solution Approach 2:
The system uses both depth cameras and regular cameras that can serve multiple functions. Depth cameras provide both hand motion detection and merchandise location information, while regular cameras provide detailed images for identification, reducing the need for specialized devices and simplifying the overall system
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for on-shelf merchandise detection are provided. One of the methods includes: obtaining a plurality of depth images associated with a shelf from a first camera; obtaining a plurality of images from one or more second cameras associated with each of a plurality of tiers of the shelf; detecting motions of a user's hand comprising reaching into and moving away from the shelf; determining one of the tiers of the shelf associated with the detected motions, a first point of time associated with reaching into the shelf, and a second point of time associated with moving away from the shelf; identifying a first image captured before the first point in time and a second image captured after the second point in time; and comparing the first image and the second image to determine one or more changes to merchandise.