Hand Image Cropping for Accurate Shopping Event Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems that rely solely on planogram data for detecting commercial events in materials handling facilities are prone to inaccuracies due to errors in planogram data, leading to incorrect tracking of item retrievals and returns.
Innovation Solution
A system that uses cameras to capture and process visual images, determine body part positions, crop images to focus on hands, filter based on event time and proximity, and generate embeddings to accurately identify items in hands, comparing these embeddings with reference images to associate events with actors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If planogram data is used to identify items involved in interactions, then the system can track item retrievals and returns, but the accuracy of event detection deteriorates due to errors in planogram data
Solution Approach 1:
The patent introduces image processing technology as an intermediary between planogram data and event detection. Cameras capture images of items on shelves, and image processing algorithms analyze these images to identify items involved in interactions. This intermediary layer validates and corrects planogram data, ensuring accurate event detection even when planogram data contains errors.
Solution Approach 2:
The patent replaces reliance on mechanical planogram data with optical detection systems. Instead of depending solely on stored planogram information, the system uses cameras and image processing to visually identify items. This substitution eliminates the accuracy limitations of planogram data while maintaining event detection functionality.
2Measurement precision
If cameras capture and process visual images to identify items in hands, then the accuracy of item identification improves, but the complexity of the system increases
Solution Approach 1:
The patent extracts and focuses on specific visual features relevant to item identification. Instead of processing entire images, the system extracts key features such as item shapes, colors, and patterns from hand-held objects. This extraction approach maintains high identification accuracy while reducing computational complexity compared to analyzing complete images.
3Measurement precision
If the system processes and filters cropped images to identify items, then the accuracy of event detection improves, but the data processing time increases
Solution Approach 1:
The patent performs preliminary actions by pre-processing and filtering images before full analysis. The system crops images to focus on hands and pre-identifies potential items of interest. This preliminary processing reduces the amount of data requiring detailed analysis, maintaining high detection accuracy while minimizing overall processing time.
Data Source
AI summary
Images captured by cameras at a store or another facility are cropped to include portions of such images depicting hands. When an event of a type is determined to have occurred at a time and at a location within the facility, the cropped images are filtered based on the type, the location, and the time of the event to include only images that might depict one item within a hand of an actor. The cropped images, as filtered, are then processed to identify the item within the hand, such as by determining embeddings or other representations of the cropped images, and comparing such embeddings or other representations to embeddings or other representations of reference images of items that are available within the facility. Based on such comparisons, an item is identified as having been taken from the facility by the customer or deposited at the facility by the customer.