Motion Model Alignment for Mixed Reality Pass-Through Imagery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Aligning camera imagery from independent devices in mixed-reality systems, such as HMDs and handheld cameras, is challenging due to positional offsets and independent movement, leading to disruptions and oscillations in composite pass-through images, which can undermine user experience with distracting artifacts.
Innovation Solution
The system determines and selects motion models based on feature correspondences from imagery captured by different cameras, generating alternative motion models for foreground and background objects at different depths, and enforces temporal consistency to reduce oscillations and improve accuracy in output imagery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If motion models are determined using feature correspondences from multiple cameras, then alignment accuracy of composite pass-through images is improved, but computational complexity and processing time increase
Solution Approach 1:
The patent segments the feature correspondence matching process into multiple independent motion model determinations - one for each camera pair combination. This allows parallel processing of feature matches from different camera perspectives, reducing the computational burden on any single processing unit while maintaining comprehensive alignment accuracy across all cameras.
Solution Approach 2:
The patent performs preliminary feature extraction and correspondence identification from all camera images before determining motion models. By pre-processing and organizing feature data in advance, the system reduces the complexity of subsequent motion model calculations, as the feature correspondence data is already structured and ready for efficient processing.
2Measurement precision
If multiple motion models are generated for different depth regions, then alignment accuracy for foreground and background objects is improved, but device complexity and processing requirements increase
Solution Approach 1:
The patent applies local quality by generating different motion models tailored to specific depth regions - foreground motion models for near objects and background motion models for distant objects. Each motion model is optimized for its specific depth range, improving alignment accuracy locally without requiring a single complex model to handle all depth variations.
Solution Approach 2:
The patent segments the scene into multiple depth regions (foreground and background) and determines separate motion models for each segment. This segmentation allows the system to process and align different depth regions independently, reducing the overall processing requirements compared to attempting to align the entire scene with a single motion model.
3Stability of the object's composition
If temporal consistency is enforced across frames, then stability of composite pass-through images is improved, but processing time and computational overhead increase
Solution Approach 1:
The patent performs preliminary determination of motion models for each frame before composite image generation. By having motion models ready in advance, the system can efficiently enforce temporal consistency during composite image creation without requiring complex real-time calculations, thus reducing processing time while maintaining image stability.
Solution Approach 2:
The patent uses feedback from previously determined motion models to guide current frame processing. By comparing current feature correspondences with previous motion model results, the system can make adjustments more efficiently and enforce temporal consistency with reduced computational overhead, as the feedback provides a reference point for minimal changes.
Data Source
AI summary
A system determining motion models for aligning scene content captured by different image sensors is configurable to access a first motion model generated based upon a set of feature correspondences that includes (i) an inlier set used to determine model parameters for the first motion model and (ii) an outlier set. The system is also configurable to define a modified set of feature correspondences that includes the outlier set from the set of feature correspondences. The system is also configurable to generate a second motion model by using the modified set of feature correspondences to determine model parameters for the second motion model.