Camera Extrinsic Calibration with 3D Point Clouds During Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing camera calibration techniques for tracking puts and takes of items in real spaces, such as shopping stores, are inefficient due to the need for frequent recalibration and disruption of operations, especially when shoppers are present, and they struggle with occlusions and camera drift.
Innovation Solution
A system and method for calibrating cameras using a 3D point cloud and neural networks to align 2D images with 3D point clouds, allowing for continuous calibration without disrupting operations by identifying immobile structures and using iterative algorithms to determine optimal camera positions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If common camera calibration techniques using special markers are used, then measurement precision is improved, but productivity deteriorates due to frequent disruption of operations for recalibration
Solution Approach 1:
The system uses shoppers themselves as calibration objects. The calibration process is self-service in nature, utilizing the natural presence of shoppers with items rather than requiring external calibration markers or personnel intervention. This eliminates the need to stop operations for recalibration while maintaining measurement precision.
Solution Approach 2:
Items held by shoppers serve as intermediaries between the camera system and the calibration process. Instead of using special markers that require setup and teardown, ordinary shopping items act as the calibration medium, allowing continuous operation while achieving accurate camera positioning and orientation.
2Adaptability or versatility
If cameras are positioned to cover large shopping areas, then adaptability is improved, but measurement precision deteriorates due to camera drift and occlusions
Solution Approach 1:
The system segments the calibration process into multiple observations across different shoppers and items. Rather than relying on a single wide-area camera view that is susceptible to drift, the system uses multiple localized calibration observations from different perspectives to collectively establish accurate camera positioning and maintain tracking precision across large areas.
3Reliability
If calibration is performed frequently to maintain accuracy, then reliability is improved, but loss of time increases due to recalibration interruptions
Solution Approach 1:
The calibration process continues uninterrupted during normal shopping operations. Shoppers naturally present calibration opportunities throughout the day, allowing the system to maintain continuous calibration without stopping operations. The useful action of calibration is embedded within the continuous flow of shopping activity rather than being a separate interruptive process.
Data Source
AI summary
Systems and techniques are provided for calibrating cameras in a real space for tracking puts and takes of items by subjects. The method includes first processing one or more selected images captured by a first plurality of cameras to extract from the images, a three-dimensional (3D) point cloud of points, and a second processing of a second set of images captured by a second plurality of cameras in the real space to match points in the second set of images to the point cloud. Differences in position of matching points can be used to determine transformation information therebetween, which can be used to calibrate the second plurality of cameras.