Shelf Item Tracking via Homography and Marker Calibration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object detection and tracking systems face challenges in real-time processing of multiple objects in busy environments, such as stores, due to computational intensity and the inability to determine physical locations of objects within images, especially when objects are removed or replaced from shelves.
Innovation Solution
A tracking system that generates homographies to map camera pixels to physical locations, allowing for object tracking across sensor fields of view, shelf position calibration, and interaction detection using markers and virtual curtains, enabling accurate assignment of items to individuals and handling of misplaced items.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional object detection and tracking systems are used in busy environments, then object identification can be achieved, but computational intensity increases significantly and real-time processing becomes difficult
Solution Approach 1:
The system segments the monitoring space into multiple zones with multiple sensors, allowing parallel processing of different spatial regions. Each sensor independently tracks objects in its field of view, and the server aggregates data from multiple sensors to achieve comprehensive tracking without requiring a single sensor to process the entire busy environment, thus reducing computational intensity while maintaining real-time capability.
2Loss of information
If conventional image-based tracking systems are used, then object positions can be identified, but the ability to determine physical locations of objects within the image space is lost
Solution Approach 1:
The system introduces markers as intermediary objects placed at known physical locations within the monitoring space. These markers serve as reference points that enable the server to correlate pixel coordinates from sensor images with actual physical locations. By detecting marker positions and using them as reference, the system recovers physical location information without requiring complex direct mapping, thus reducing information loss while maintaining manageable system complexity.
3Area of stationary object
If multiple sensors are deployed to cover large physical spaces, then comprehensive object tracking is achieved, but the complexity of mapping pixel locations to physical spaces increases
Solution Approach 1:
The system applies local quality by creating individual homography mappings for each sensor's field of view rather than attempting a single global mapping. Each sensor zone is calibrated independently using markers visible to that specific sensor, allowing the system to handle large physical spaces with multiple sensors while keeping the mapping complexity localized to each sensor's view. This modular approach reduces overall system complexity compared to a unified mapping approach.
Data Source
AI summary
An image sensor is positioned such that a field-of-view of the image sensor encompasses at least a portion of a structure configured to store items. The image sensor generates angled-view images of the items stored on the structure. A tracking subsystem determines that a person has interacted with the structure and receives image frames of the angled-view images. The tracking subsystem determines that the person interacted with a first item stored on the structure. A first image is identified associated with a first time before the person interacted with the first item, and a second image is identified associated with a second time after the person interacted with the first item. If it is determined, based on a comparison of the first and second images, that the item was removed from the structure, the first item is assigned to the person.