Multi-Camera Image Processing for Customer Distance Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video stream analysis systems in retail environments fail to accurately estimate distances between objects, track customer journeys, and distinguish between static and dynamic objects, leading to incomplete understanding of store traffic patterns and interactions.
Innovation Solution
A computer-implemented method that processes video streams from multiple cameras, normalizes frames, extracts bounding boxes, merges them into tracklets, and estimates distances using feature vectors and interpolation, while removing false positives and detecting staff interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If video streams from multiple cameras are used to estimate distances between objects, then distance estimation capability is improved, but measurement precision deteriorates due to non-uniform floor distance distribution from pixel to pixel across different camera views
Solution Approach 1:
The system transforms the non-uniform pixel-to-pixel distance relationship into a uniform metric by introducing a calibrated distance metric that accounts for camera-specific distortion and viewing angles. This allows consistent distance measurement across different camera views by converting pixel coordinates to physical distance units using camera calibration parameters.
Solution Approach 2:
A camera calibration model acts as an intermediary between the camera's pixel coordinate system and the physical world coordinate system. This calibration model includes intrinsic parameters (focal length, principal point) and extrinsic parameters (rotation, translation) that mediate the transformation, enabling accurate distance estimation by bridging the non-uniform pixel distances with uniform physical distances.
2Measurement precision
If bounding boxes are extracted from normalized frames to generate feature vectors, then object detection accuracy is improved, but device complexity increases due to the multi-step processing pipeline
Solution Approach 1:
The processing pipeline is segmented into distinct functional modules: normalization module, bounding box extraction module, feature vector generation module, and distance calculation module. Each module performs a specific transformation and passes results to the next module, making the complex pipeline modular and manageable while maintaining high detection accuracy through specialized processing at each stage.
3Productivity
If tracklets are merged to generate analytical outputs, then customer journey tracking is improved, but loss of information increases due to missing data from cameras
Solution Approach 1:
The system implements feedback mechanisms where detected customer trajectories and camera coverage information are continuously fed back to optimize the tracking algorithm. When data is missing from certain cameras, the feedback loop adjusts the trajectory prediction models to better utilize available data from other cameras, reducing information loss while maintaining comprehensive customer journey tracking.
Solution Approach 2:
The system performs partial tracking using available camera data and supplements it with predictive models that infer missing information based on spatial relationships and movement patterns. This partial action approach allows the system to operate effectively even with incomplete data from some cameras, maintaining productivity while minimizing information loss.
Data Source
AI summary
Aspects of the example implementations are directed to methods, systems and apparatuses associated with processing of images associated with objects in a view of one or more cameras co-located in a space. The example implementations may be directed to estimating a physical distance between a first person and a second person (e.g., customers) in a camera view; detecting an interest in a section (e.g., shelf) by the first person or second person; estimating relative distances between views of the one or more cameras, to determine which camera views are overlapping and non-overlapping; and detecting static objects (e.g., false positives) that may appear similar to a customer, such as posters of people, mannequins, or the like, and removing the static objects from the processed images.