Dynamic Multi-Camera Calibration Using 3D Joint Feature Points
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing camera calibration methods for multiple cameras, such as structure-from-motion (SfM) and using a checkerboard, are limited by the requirement that 2D feature points be on the same plane or necessitate a physical checkerboard, and fail to account for camera movement, leading to inaccuracies in estimating positional relationships.
Innovation Solution
An electronic device extracts 2D joint feature points from multiple camera images, lifts them to 3D, and predicts a positional relationship between cameras using a projection relationship without the need for a checkerboard or fixed 3D reference points, enabling accurate calibration even with moving cameras.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If structure-from-motion (SfM) is used for camera calibration, then 2D feature points can be extracted from images, but the method is limited to pairs of 2D feature points on the same plane and cannot be applied to other planes
Solution Approach 1:
The patent transforms 2D feature points into 3D feature points by introducing depth information through time-based motion analysis. By tracking the movement of feature points across multiple frames and using the known camera motion, the system reconstructs 3D positions, thereby extending the calibration method from plane-limited 2D space to full 3D space.
2Measurement precision
If a checkerboard is used for camera calibration, then grid points can be matched between images, but the method requires all grid points to be on the same plane and requires re-preparation when camera position changes
Solution Approach 1:
The patent enables the environment itself to serve as the calibration object. Instead of requiring an external checkerboard, the system uses naturally occurring 3D features in the scene and the camera's own motion to perform calibration. The camera calibration is achieved through self-observation of feature point movements in the environment.
Solution Approach 2:
The patent extracts calibration information directly from video sequences by identifying and tracking feature points across frames. The essential calibration data is extracted from natural scene content rather than requiring a separate calibration artifact, eliminating the need for physical checkerboards.
3Measurement precision
If camera calibration is performed using fixed reference points, then positional relationships can be estimated, but the method fails to account for camera movement in mobile devices
Solution Approach 1:
The patent transitions from static calibration using fixed reference points to dynamic calibration that explicitly models camera motion. By tracking feature point movements over time and incorporating camera motion parameters, the system achieves calibration that is inherently adapted to moving cameras, making it suitable for mobile devices.
Data Source
AI summary
An electronic device for performing calibration is provided. The electronic device includes a communication interface, memory storing one or more computer programs, and one or more processors communicatively coupled to the communication interface and the memory, wherein the one or more computer programs include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to obtain, via the communication interface, a first image of a user captured by a first camera, and a second image of the user captured by a second camera, extract first joint feature points, which are two-dimensional (2D) position coordinate values of joints of the user, from the first image, and second joint feature points, which are 2D position coordinate values of the joints, from the second image, obtain three-dimensional (3D) joint feature points of the joints by lifting the extracted first joint feature points to 3D position coordinate values, obtain a projection relationship for projecting the 3D joint feature points onto the 2D position coordinate values of the second joint feature points, and perform camera calibration by predicting a positional relationship between the first camera and the second camera based on the obtained projection relationship.