AR Proxy Object Overlay With Dynamic Occlusion Anchoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AR systems fail to align virtual models accurately with physical proxy objects, leading to protrusions, misalignment of edges, and occlusion issues, resulting in a sub-optimal user experience and glitches when interacting with virtual content.
Innovation Solution
An AR system that detects features on physical and occluding objects, anchors virtual content to both, and adjusts alignment dynamically to maintain visibility and alignment during interactions, using machine learning techniques for image inpainting to conceal occluded portions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a virtual model is overlaid over a physical proxy object to improve user experience, then user immersion is enhanced, but misalignment between the virtual model and proxy object causes tactile feedback delays and breaks immersion
Solution Approach 1:
The system dynamically adjusts the virtual model's position and orientation to match the proxy object's movements in real-time. The AR system continuously tracks the proxy object's six-degree-of-freedom motion and updates the virtual model's transformation matrix accordingly, ensuring dynamic alignment that adapts to user interactions and maintains immersion without misalignment artifacts.
Solution Approach 2:
The system uses visual feedback to detect and correct misalignment between the virtual model and proxy object. By monitoring the relative positions and orientations, the system automatically adjusts the virtual model's placement to eliminate protrusions and alignment errors, ensuring the virtual content remains precisely registered on the physical object during user interactions.
2Measurement precision
If the virtual model is made to match the proxy object's dimensions, then alignment improves, but portions of the proxy object may become hidden when users physically interact with the virtual model
Solution Approach 1:
The system introduces a virtual hand or occlusion indicator as an intermediary element to represent the user's physical interaction. When a user's hand occludes the proxy object, the system renders a corresponding virtual hand that interacts with the virtual model, allowing the actual proxy object to remain visible while still conveying the interaction information to the user.
Solution Approach 2:
The system uses transparency and opacity changes to manage visibility. When the proxy object needs to be partially obscured to maintain alignment, the system adjusts the virtual model's transparency or adds visual indicators to show that the object is still present and interactable, preventing complete loss of information about the proxy object's state and position.
3Measurement precision
If the AR system continuously tracks the proxy object to maintain alignment during user interactions, then alignment accuracy is maintained, but computational complexity and processing requirements increase
Solution Approach 1:
The system segments the tracking task into distinct components: initial proxy object detection and feature extraction, continuous motion tracking using inertial sensors, and virtual model transformation calculation. By dividing the complex tracking problem into manageable segments, the system maintains high alignment accuracy while reducing overall computational complexity through specialized processing for each segment.
Solution Approach 2:
The system performs preliminary action by pre-processing the proxy object detection and establishing the initial transformation matrix before user interactions begin. This includes detecting the proxy object's position, orientation, and dimensions in advance, so that during interactions, the system only needs to track motion changes rather than perform full object recognition, significantly reducing real-time computational requirements.
Data Source
AI summary
To improve user experience when interacting with AR content within an AR environment, the AR content may be overlaid over a proxy object in a real-world space. However, issues such as the AR content and the proxy object not being aligned, or occlusion of the proxy object leading to glitching of the AR content, may decrease user enjoyment. In some embodiments, an AR system for the overlay of AR content may be implemented. The system may anchor a virtual model to the proxy object based on detected features on the proxy object. The anchoring may include aligning elements of the virtual model and proxy object. In response to an occluding object occluding some features on the proxy object, the system may anchor the virtual model to the occluding object, or to both the proxy object and the occluding object, based on detected features on the occluding object.