Inventory Action Detection Using Overlapping Camera Views
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing systems face challenges in accurately identifying and tracking 'put and take' actions of subjects in large spaces, such as shopping stores, due to occlusions and the complexity of processing multiple camera feeds, which hinders efficient inventory management.
Innovation Solution
A system utilizing multiple cameras with overlapping fields of view, coupled with convolutional neural networks, processes image sequences to identify subjects, locate joints, and classify actions like putting or taking items, without the need for retrofitting shelves or floor spaces, using synchronized image recognition engines and circular buffers to generate reliable logs of inventory actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If multiple cameras are used to cover large spaces, then the coverage area is improved, but the complexity of processing multiple camera feeds increases
Solution Approach 1:
The patent divides the large space into multiple overlapping fields of view, each captured by a separate camera. The processing system segments the overall task into individual camera feed processing, subject identification in each segment, and then integrates results across segments. This segmentation allows parallel processing and reduces the complexity of handling the entire large space at once.
Solution Approach 2:
The patent merges information from multiple camera feeds by processing images from different cameras simultaneously and integrating subject tracking across camera boundaries. The system combines data from overlapping fields of view to create a unified understanding of subject movements and actions across the entire monitored area, resolving the complexity through coordinated processing.
2Measurement precision
If images from multiple cameras are processed to track subjects, then the tracking accuracy is improved, but the time required for processing increases
Solution Approach 1:
The patent performs preliminary processing by pre-identifying subjects and their locations in individual camera feeds before integrating across cameras. The system pre-processes each camera feed to extract subject information, bounding boxes, and action classifications, then uses these pre-computed results for rapid integration and tracking, reducing the overall processing time while maintaining accuracy.
Solution Approach 2:
The patent maintains continuous processing of camera feeds with overlapping fields of view, ensuring that subject tracking is uninterrupted. The system continuously processes new frames while maintaining context from previous frames, enabling real-time tracking without significant processing delays. The overlapping fields of view provide continuous coverage that eliminates gaps in tracking.
3Speed
If subjects are tracked in real-time, then the responsiveness is improved, but the reliability of action detection decreases due to occlusions
Solution Approach 1:
The patent implements feedback mechanisms where the system continuously monitors subject actions and adjusts its detection algorithms based on observed patterns. When occlusions are detected or ambiguous actions are identified, the system uses feedback to refine its classification and re-evaluate potential actions, improving reliability while maintaining real-time responsiveness through iterative processing.
Solution Approach 2:
The patent addresses occlusion problems by adding temporal dimension to the analysis. Instead of relying solely on single-frame images, the system processes sequences of images across time, allowing it to infer actions even when subjects are partially occluded. This temporal dimension provides additional information that improves action detection reliability without sacrificing real-time processing.
4Ease of manufacture
If no sensors are installed on shelves and floor, then the ease of deployment is improved, but the measurement precision of inventory actions decreases
Solution Approach 1:
The patent replaces mechanical sensors installed on shelves and floors with an optical system using overhead cameras. Instead of using physical contact sensors that require retrofitting, the system uses image processing and computer vision algorithms to detect and track inventory actions. This substitution maintains or improves measurement precision while dramatically simplifying deployment, as cameras can be installed overhead without modifying the store infrastructure.
Data Source
AI summary
Systems and techniques are provided for tracking puts and takes of inventory items by subjects in an area of real space. A plurality of cameras with overlapping fields of view produce respective sequences of images of corresponding fields of view in the real space. A processing system is coupled to the system. In one embodiment, the processing system comprises image recognition engines receiving corresponding sequences of images from the plurality of cameras. The image recognition engines process the images in the corresponding sequences to identify subjects represented in the images and generate classifications of the identified subjects. The system processes the classifications of identified subjects for sets of images in the sequences of images to detect takes and puts of inventory items on shelves by identified subjects.