Motion Model Selection for Mixed Reality Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mixed-reality systems face challenges in accurately aligning camera imagery from separate device cameras with HMD cameras, leading to disruptions and oscillations in pass-through images due to changes in feature contributions from objects at different depths, affecting user experience.
Innovation Solution
The system generates multiple motion models based on different subsets of feature correspondences from imagery captured by HMD and separate cameras, selecting a final motion model that enforces temporal consistency and focuses on objects of interest to reduce oscillations and improve alignment accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple motion models are generated based on different subsets of feature correspondences, then alignment accuracy is improved, but device complexity increases
Solution Approach 1:
The patent segments the feature correspondences into different subsets based on depth information, creating multiple motion models (first motion model from first subset, second motion model from second subset) to handle objects at different depths separately. This segmentation allows each motion model to focus on specific depth ranges, improving overall alignment accuracy while managing complexity through organized division of processing tasks.
Solution Approach 2:
The patent changes the parameter of feature correspondence selection by using depth information to divide features into different subsets. By varying which features are included in each subset based on their depth characteristics, the system generates multiple motion models with different parameter configurations, thereby improving alignment precision for objects at various depths.
2Speed
If motion models are updated frequently to track scene changes, then responsiveness is improved, but temporal consistency deteriorates
Solution Approach 1:
The patent performs preliminary actions by generating multiple motion models in advance based on different feature subsets before selecting the final model. This preliminary generation of multiple models allows the system to have ready options that maintain temporal consistency while still being responsive to scene changes, as the selection process can choose from pre-computed models rather than generating a new model from scratch.
Solution Approach 2:
The patent implements feedback by comparing the first and second motion models to select a final motion model that maintains temporal consistency. The system uses feedback from the comparison process to determine which motion model best represents the current scene while maintaining consistency with previous frames, thereby balancing responsiveness with temporal stability.
3Quantity of substance
If all feature correspondences are used to generate motion models, then comprehensive scene coverage is improved, but measurement precision deteriorates
Solution Approach 1:
The patent segments the large quantity of feature correspondences into multiple subsets based on depth information. Instead of using all features uniformly, the system divides them into first and second subsets, each processed to create separate motion models. This segmentation improves measurement precision by ensuring that features at similar depths are processed together, reducing errors from depth-related perspective differences.
Solution Approach 2:
The patent applies local quality by treating different depth regions with different processing approaches. Each motion model is specialized for specific depth ranges, with local optimization for those regions. This allows the system to maintain high precision locally for each depth subset while still achieving comprehensive scene coverage through the combination of multiple specialized models.
Data Source
AI summary
A system for selecting motion models for aligning scene content captured by different image sensors, is configurable to (i) access a first image captured by a first image sensor and a second image captured by a second image sensor; (ii) access a set of motion models; (iii) define a reference patch within the second image; (iv) generate a respective match patch for each motion model of the set of motion models; (v) determine a similarity between each respective match patch and the reference patch within the second image; (vi) select a final motion model from the set of motion models based upon the similarity between each respective match patch and the reference patch within the second image; and (vii) utilize the final motion model to generate an output image for display to a user.