Camera Frame Correspondence Search Using Overlap Region Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for aligning images from separate cameras in mixed-reality systems are computationally expensive, particularly when searching for feature points over large search areas, necessitating improved methods for continuous image alignment.
Innovation Solution
A computing system reduces the search area for identifying correspondences between reference and match frames by using transformation matrices based on inertial tracking data from IMUs and alignment matrices, intelligently determining a subset of pixels for feature matching, and updating alignment matrices based on visual correspondences and confidence measures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If feature extraction and feature matching is performed over large search areas to accurately align images from separate cameras, then alignment precision is improved, but computational cost increases significantly
Solution Approach 1:
The patent divides the search area into multiple regions of interest (ROIs) based on feature point locations. Instead of performing exhaustive feature matching over the entire large search area, the system segments the search space into smaller manageable regions around detected feature points, significantly reducing computational requirements while maintaining alignment precision.
Solution Approach 2:
The system performs preliminary feature extraction and feature point detection before conducting the full feature matching process. By pre-identifying key feature points and their locations in both camera views, the system prepares the data structure in advance, enabling more efficient and targeted matching operations that reduce overall computational cost.
2Productivity
If continuous image alignment is performed in real-time for composite pass-through images, then user experience is improved, but processing time requirements increase
Solution Approach 1:
The patent implements periodic feature extraction and matching at specific time intervals or frame rates rather than continuous processing. By updating alignment information periodically based on camera motion detection and scene change analysis, the system maintains real-time alignment quality while reducing processing time requirements compared to continuous frame-by-frame processing.
Solution Approach 2:
The system dynamically adjusts the frequency and intensity of alignment processing based on detected motion levels, scene complexity, and computational resource availability. When motion is detected or scene changes occur, the system increases processing frequency; during stable periods, it reduces processing intensity, enabling real-time performance with optimized processing time.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system for reducing a search area for identifying correspondences identifies an overlap region within a first match frame captured by a match camera. The overlap region includes one or more points of the first match frame that are associated with one or more same portions of an environment as one or more corresponding points of a first reference frame captured by a reference camera. The system obtains a second reference frame captured by the reference camera and a second match frame captured by the match camera. The system identifies a reference camera transformation matrix, and/or a match camera transformation matrix. The system defines a search area within the second match frame based on the overlap region and the reference camera transformation matrix and/or the match camera transformation matrix.