AR Proxy Object Overlay With Occlusion Concealment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AR systems fail to align virtual models accurately with physical proxy objects, leading to protrusions, misalignment of edges and surfaces, and occlusion issues that disrupt the user's immersive experience and cause glitches or disappearance of the virtual model.
Innovation Solution
An AR system that detects features on physical and occluding objects, anchors virtual content to these objects, and adjusts alignment to maintain visibility and tactile feedback, using techniques like image inpainting to conceal occluded portions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If the virtual model is overlaid over the physical proxy object to allow tactile feedback, then user interaction realism is improved, but alignment accuracy deteriorates causing protrusions and misalignment
Solution Approach 1:
The system dynamically adjusts the virtual model's position and orientation based on real-time detection of the proxy object's features. The virtual model is not statically positioned but continuously repositioned to maintain alignment with the proxy object, resolving the contradiction between interaction realism and alignment accuracy.
Solution Approach 2:
The system uses feedback from feature detection algorithms to monitor the proxy object's position and orientation, then adjusts the virtual model accordingly. This closed-loop control ensures the virtual model remains aligned with the proxy object during user interaction, maintaining both realism and precision.
2Ease of operation
If the proxy object is made larger to improve tactile feedback, then interaction comfort is improved, but visual occlusion worsens causing portions to protrude
Solution Approach 1:
The system transitions from a 2D overlay problem to a 3D spatial solution by detecting depth and orientation of the proxy object. This allows the virtual model to be positioned in three-dimensional space above the proxy object, preventing visual protrusion while maintaining the proxy object's size for comfortable interaction.
Solution Approach 2:
The system changes parameters such as the virtual model's elevation height, rotation angle, and scale based on the proxy object's detected features. By adjusting these parameters dynamically, the system maintains full visibility of the proxy object while preserving interaction comfort.
3Measurement precision
If the virtual model is anchored to the proxy object using detected features, then tracking accuracy is improved, but system complexity increases due to feature detection and occlusion handling
Solution Approach 1:
The system segments the proxy object into distinct detectable features such as edges, corners, and surfaces. By identifying and tracking these individual features separately, the system achieves high tracking accuracy while managing complexity through modular feature detection rather than processing the entire object simultaneously.
Solution Approach 2:
The system introduces an intermediary layer of feature detection and matching algorithms that bridge the proxy object and virtual model. This intermediary processing layer simplifies the overall system by breaking down the complex task of tracking into manageable feature-level operations, reducing computational complexity while maintaining accuracy.
4Ease of operation
If the user physically interacts with the virtual model by moving the proxy object, then interaction immersion is improved, but occlusion detection accuracy deteriorates when features are hidden
Solution Approach 1:
The system performs preliminary feature detection and stores feature data before occlusion occurs. By pre-processing and caching feature information, the system maintains tracking accuracy even when features become temporarily occluded during user interaction, resolving the contradiction between immersion and detection accuracy.
Solution Approach 2:
The system prepares backup tracking methods and predicts feature positions in advance to cushion against occlusion. When occlusion occurs, the system can rely on predicted positions or alternative features, maintaining tracking accuracy without disrupting user immersion.
Data Source
AI summary
To improve user experience when interacting with AR content within an AR environment, the AR content may be overlaid over a proxy object in a real-world space. Differences in dimension between the proxy object and the virtual model may be such that the object is larger than the virtual model, which may result in portions of the object appearing to protrude from behind the virtual model, decreasing user enjoyment. In some embodiments, an AR system for the overlay of AR content on a proxy object and concealment of the proxy object may be implemented. The system may overlay a virtual model to a proxy object, and then conceal any remaining visible portions of the proxy object from the visual field of a device displaying the AR environment. The system may overlay the virtual model so that any remaining visible portion of the proxy object is a single continuous region.