Multi-camera extrinsic calibration using movement track association
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-camera systems face challenges in extrinsic calibration when multiple individuals or objects are present in the field of view, leading to errors in keypoint association and inefficient image acquisition processes.
Innovation Solution
A computer-implemented method for extrinsic calibration that processes image data from multiple cameras to detect and track movement of multiple objects, associating keypoints across views to generate calibration data, using techniques such as Structure from Motion algorithms and bundle adjustment to refine camera positions and orientations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing extrinsic calibration techniques are used with multiple cameras, then calibration data can be obtained, but errors occur in keypoint association when multiple individuals are present in the field of view
Solution Approach 1:
The patent segments the calibration process into distinct phases: detecting individual objects in each camera view, tracking their movement over time, and then associating detected objects across cameras based on their movement trajectories. This segmentation allows the system to handle multiple objects by processing them individually through defined stages, preventing confusion between objects from different individuals.
Solution Approach 2:
The patent performs preliminary object detection and movement tracking within each camera view before attempting cross-camera association. By pre-identifying and tracking objects in each view separately, the system establishes reliable movement patterns that serve as the basis for subsequent association, ensuring accurate keypoint matching even when multiple individuals are present.
2Productivity
If a single individual is used for extrinsic calibration, then the process can be automated, but the image acquisition efficiency is reduced due to fewer keypoints available
Solution Approach 1:
The patent converts the potential harm of multiple objects causing association errors into a benefit by using movement tracking to uniquely identify and associate objects from different individuals across cameras. The system leverages the presence of multiple moving objects to increase the number of available keypoints for calibration, thereby improving image acquisition efficiency while maintaining accuracy through motion-based discrimination.
3Measurement precision
If movement tracking is performed to handle multiple objects, then keypoint association accuracy improves, but the processing complexity increases
Solution Approach 1:
The patent implements a universal object detection and tracking framework that handles multiple objects simultaneously using the same detection and tracking algorithms. This multi-functional approach processes all detected objects through a unified pipeline, reducing the need for object-specific handling logic and managing processing complexity through standardized procedures.
Data Source
Figure 1A~1D
Figure 2~3A
Figure 3B~4B
AI summary
A device performs extrinsic calibration of a multi-camera system comprising first and second cameras. The device obtains image sequences [I1], [I2] taken by the respective camera, processes [I1], [I2] for detection of first and second movement tracks [T1], [T2], and evaluates [T1] in relation to [T2] to determine candidate track pairs, [CTP], each candidate track pair in [CTP] being deemed to originate from a common object. The device processes [CTP] to determine calibration items [CI], with each calibration item in [CI] representing a relative position between the cameras, and operates [CI] on a respective candidate track pair in [CTP] to generate match scores [MS]_i. The device aggregates match scores in [MS]_i to determine a total score TS_i for the respective candidate track pair, determines final track pairs [FTP] among [CTP] based on TS_i for the respective candidate track pair, and determines final calibration data FCD that represents the relative position between the cameras based on [FTP].