Hand-Shelf Interaction Detection Using Region-of-Interest Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in efficiently identifying and tracking multiple objects in real-time, especially in large physical spaces, due to computational intensity and the need for significant time processing images. Additionally, these systems struggle to determine the physical location of objects within images and handle the movement of objects between the fields of view of different sensors.
Innovation Solution
The system generates a relationship between camera pixels and physical locations within a space by using homographies to map pixel locations to global coordinates. It also employs processes for handing off tracking information between sensors, detecting shelf interactions, associating items with persons, and identifying misplaced items based on weight and location.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If image processing techniques are used to identify and track objects in real-time, then object tracking capability is improved, but computational intensity increases and processing time extends
Solution Approach 1:
The system divides the large physical space into multiple zones, each monitored by dedicated sensors. Objects are tracked segment-by-segment as they move through zones, rather than processing entire large-space images simultaneously. This segmentation reduces computational burden per processing cycle while maintaining continuous tracking capability.
Solution Approach 2:
The system pre-establishes zone boundaries, sensor placements, and tracking parameters before objects enter the monitoring space. Homographies and coordinate transformations are pre-computed for each zone, allowing objects to be tracked immediately upon entry without delay for setup or configuration during active tracking.
2Area of stationary object
If multiple sensors are deployed to cover large physical spaces, then spatial coverage is improved, but handling object movement between sensor fields of view becomes more complex
Solution Approach 1:
The system merges tracking information from multiple sensors by establishing a global coordinate system that integrates local sensor coordinates. Homographies map each sensor's field of view to the global coordinate system, allowing seamless combination of tracking data across sensors. This merging approach handles object movement between sensors automatically through coordinate transformation rather than complex inter-sensor communication.
Solution Approach 2:
The global coordinate system acts as an intermediary that mediates between multiple local sensor coordinate systems. Instead of sensors directly communicating with each other to track objects crossing boundaries, all tracking information is transformed through the global coordinate system, which serves as a neutral mediator that simplifies the handoff process between sensors.
3Measurement precision
If homographies are used to map pixel locations to global coordinates, then location determination accuracy is improved, but computational requirements increase
Solution Approach 1:
Homographies and coordinate transformation matrices are pre-computed and stored for each sensor and zone boundary before tracking begins. When objects are tracked, the system simply applies these pre-computed transformations rather than calculating them in real-time, dramatically reducing computational energy requirements while maintaining high location accuracy.
Data Source
AI summary
An image sensor is positioned such that a field-of-view of the image sensor encompasses at least a portion of an object storing items. The image sensor generates angled-view images of an object. A pixel position of a body part of a person is determined in at least a subset of the received image frames, thereby determining a set of pixel positions of the body part. An aggregated body part position is determined based on the set of pixel positions. If the aggregated body part position is determined to correspond to a position associated with the object, a trigger signal is provided indicating an interaction event has occurred.