AR Anchor Pose Resolution Using Video Features and CAD Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Marker-based and model-based anchor approaches for Augmented Reality (AR) applications are costly, time-consuming, and prone to latency due to the need for physical environment labeling and point-cloud model mapping, which are inadequate for complex objects.
Innovation Solution
A method and system that utilizes feature detection from a video feed, mapping these features to 2D and 3D arrays, and applying a Perspective-n-Point (PnP) algorithm to estimate and dynamically track the anchor object's pose, eliminating the need for physical markers and point-cloud models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If marker-based anchor approaches are used, then anchor identification is achieved, but manufacturing costs increase and physical environment labeling is required
Solution Approach 1:
The patent extracts the anchor identification function from physical markers and point-cloud models, implementing it through feature detection algorithms that automatically identify anchor objects in video feeds without requiring physical labeling or pre-existing 3D models
Solution Approach 2:
The system enables self-service anchor detection by automatically identifying and resolving anchor objects using computer vision algorithms, eliminating the need for manual marker placement or user interaction for model alignment
2Reliability
If point-cloud model mapping is used, then anchor identification is achieved, but session startup latency increases due to additional user interaction
Solution Approach 1:
The patent performs preliminary feature extraction and anchor identification during the AR session initiation phase using video feed analysis, establishing the anchor pose before the actual AR content is displayed, thereby reducing session startup latency
Solution Approach 2:
The system replaces the mechanical interaction-based model alignment process with automated computer vision algorithms that detect and resolve anchor objects through feature matching and pose estimation in video feeds
3Device complexity
If single model representation is used, then model simplicity is maintained, but it cannot adequately represent different configurations of complex physical objects
Solution Approach 1:
The patent implements dynamic anchor resolution by continuously tracking and updating anchor pose information during the AR session, allowing the system to adapt to different object configurations and states rather than relying on a single static model representation
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Interesting features of an object within a physical environment are identified from a video of the physical environment. A model of the object is obtained, the model includes dimensions of the object. A first array is maintained with the pixel coordinates of the identified features within the video. A second array is maintained with the dimensions of the features obtained from the model. A processing framework is provided for using the first and second arrays and estimating, updating, and tracking the anchor pose in the physical environment within the video along with any AR generated objects placed in the video. The framework provided to an Augmented Reality (AR) app to initiate an AR session and to maintain the session.