Online Projector Camera Calibration Using Feature Correspondence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing projector-camera calibration methods require a separate and offline calibration step, which is inconvenient for real-time systems and those with large projector-camera baselines, as they involve projecting structured-light patterns or checkerboards, causing delays before desired content can be projected.
Innovation Solution
A fully automated online projector-camera calibration method using natural images or structured-light patterns with distinguishable features, where a correspondence map is created between projector and camera images, allowing for iterative estimation of intrinsic, extrinsic, and distortion parameters, enabling continuous operation without separate calibration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If offline calibration with structured-light patterns or checkerboards is performed, then calibration accuracy is improved, but system operation time is increased due to separate calibration step
Solution Approach 1:
The patent merges the calibration function with the normal projection function by projecting natural images that contain distinguishable features instead of dedicated calibration patterns. This allows the system to perform calibration continuously during operation rather than requiring a separate offline calibration step, thus reducing system operation time while maintaining calibration accuracy through feature-based correspondence matching.
Solution Approach 2:
The calibration process becomes continuous rather than discrete by using natural images with distinguishable features that can be processed during normal system operation. The system continuously establishes and updates correspondence maps between projector and camera images, allowing calibration to occur throughout system usage rather than requiring a dedicated calibration phase, thereby eliminating idle time.
2Measurement precision
If structured-light patterns are projected for calibration, then calibration precision is improved, but viewer experience is degraded due to exposure to calibration patterns
Solution Approach 1:
The projected images serve dual functions: they act as both calibration patterns and useful content for viewers. By using natural images with distinguishable features (such as images with text, logos, or high-contrast elements), the system maintains calibration precision while simultaneously providing meaningful visual content, thus eliminating the need to switch between calibration and content projection modes.
Solution Approach 2:
The patent changes the parameters of the projection content from dedicated calibration patterns (grayscale codes, binary patterns) to natural images with distinguishable features. This parameter change allows the projection to serve both calibration and content delivery purposes simultaneously, improving viewer experience by eliminating exposure to monotonous calibration patterns while maintaining calibration precision through feature detection and correspondence mapping.
3Speed
If feature extraction techniques are used to match distinguishable features, then correspondence mapping speed is improved, but mapping completeness is reduced due to sparse correspondences
Solution Approach 1:
The patent employs an iterative feedback mechanism where the system projects images, captures them with the camera, extracts features, establishes correspondences, refines the calibration parameters, and repeats the process. This feedback loop allows the system to progressively improve mapping completeness by using the calibration results to guide subsequent feature matching, ensuring that even initially sparse correspondences converge to accurate and complete mappings over multiple iterations.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system and method for online projector-camera calibration from one or more images is provided. The system (200) comprises: a projector (207), a camera (214), and a calibration device (201) configured to: determine a map (560) between pixels of each of a projector image (440) and a camera image (450) in fewer than one hundred percent of pixels of the projector image (440) using feature extraction; determine an initial estimate of a fundamental matrix from the map (560); determine an initial guess of intrinsic properties of the projector (207) and camera (214) using one or more closed-form solutions; iteratively determine an error-function based on the map (560) using the fundamental matrix while adding constraints on the intrinsic properties using the one or more closed-form solutions, and the initial estimate and guess as initial input; and, when the error-function reaches an acceptance value, determine intrinsic and extrinsic properties of the projector (207) and the camera (214) from current values of iterative estimates.