AR Occlusion via Pre-generated 3D Pass-through Textures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality (AR) systems often occlude physical objects with virtual content, reducing realism by making it appear as if virtual elements are closer to the user than they actually are, due to inadequate handling of peripheral device locations and orientations.
Innovation Solution
The method involves determining the location and orientation of peripheral devices in a real-world environment, generating a pass-through texture using pre-generated models, and inserting this texture into virtual content to allow real-world light to pass through without being occluded by virtual elements, ensuring that physical objects are visible and maintaining realistic spatial relationships.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If virtual content is displayed in AR system, then virtual elements can be overlaid on real-world view, but physical objects are occluded by virtual content reducing realism
Solution Approach 1:
The system pre-generates 3D models of peripheral devices before the AR session begins. These models are stored and ready to be used when needed for occlusion handling, allowing the system to prepare occlusion data in advance rather than computing it in real-time during the AR experience.
Solution Approach 2:
The patent introduces pass-through textures as an intermediary layer between the virtual content and the real-world view. These textures allow light from physical objects to pass through virtual elements, creating a mediator that preserves visibility of real-world objects while maintaining the AR overlay effect.
2Productivity
If pass-through texture is generated using pre-generated model, then rendering speed is improved, but model accuracy may be reduced compared to real-time capture
Solution Approach 1:
The system creates accurate 3D copies (models) of peripheral devices that can be reused multiple times. These digital models replicate the physical devices' geometry and properties, allowing the system to use them for generating pass-through textures without needing to capture or process real device data during the AR session.
Solution Approach 2:
The 3D models of peripheral devices are generated and stored beforehand, before the AR application runs. This preliminary modeling work eliminates the need for real-time device scanning or modeling, significantly improving rendering speed while maintaining sufficient accuracy for occlusion purposes.
Data Source
AI summary
To prevent virtual content from occluding physical objects in an augmented reality (AR) system, the embodiments herein describe generating a pass-through texture using a pre-generated model of a physical object. In one embodiment, the AR system determines the location of the physical object as well as its orientation. With this information, the system rotates and re-sizes the pre-generated model of the object to match its location and orientation in the real-world. The system then generates the pass-through texture from the pre-generated model and inserts the texture into the virtual content. When the virtual content is displayed, the pass-through texture permits light from the real-world view to pass through substantially unaffected by the virtual content. In this manner, the physical object (which aligns with the location of the pass-through texture in the virtual content) can be seen by the user—i.e., is not occluded by the virtual content.