Occlusion Rendering Using Live Depth and Raycast 3D Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR), mixed reality (MR), and virtual reality (VR) systems face significant challenges in providing realistic experiences due to high computational demands for real-time 3D world reconstruction, resource consumption, and the need for accurate positioning of virtual objects relative to the user's environment, which often requires substantial processing power and memory.
Innovation Solution
The system reduces computational resources by representing the physical world in blocks that can be stored and retrieved separately, using simplified data formats like depth maps and meshes, and selectively accessing more detailed representations only when necessary, allowing for efficient updating and rendering of virtual objects in real-time with limited computational overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If real-time 3D world reconstruction is performed with high detail, then rendering accuracy and realism are improved, but computational resource consumption increases
Solution Approach 1:
The patent divides the 3D world representation into discrete blocks that can be independently processed, stored, and retrieved. This segmentation allows the system to handle only relevant portions of the environment at any given time, reducing overall computational resource consumption while maintaining rendering accuracy for visible areas.
Solution Approach 2:
The system applies different levels of detail to different blocks based on their relevance to the user's current view. Blocks within the user's field of view are processed with high detail, while distant or occluded blocks use lower detail representations, optimizing the balance between rendering accuracy and computational resources.
2Measurement precision
If complete 3D world reconstruction is maintained, then virtual object positioning accuracy is improved, but memory requirements increase
Solution Approach 1:
The world is divided into discrete blocks that can be independently stored and managed in memory. This allows the system to load only the blocks currently needed for accurate virtual object positioning, maintaining precision where required while reducing overall memory requirements through selective loading and unloading of block data.
Solution Approach 2:
The system pre-processes and stores world information in a block-based format with embedded surface information, allowing for efficient retrieval and processing when virtual objects need to be positioned accurately. This preliminary organization of data reduces the computational and memory burden during runtime operations.
3Manufacturing precision
If detailed surface information is continuously updated, then occlusion rendering accuracy is improved, but processing time increases
Solution Approach 1:
Surface information is associated with discrete blocks rather than the entire world model. This segmentation allows the system to update and process surface information only for relevant blocks, improving occlusion rendering accuracy for visible areas while significantly reducing the processing time required compared to continuous full-world updates.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An augmented reality/mixed reality system that provides a more immersive user experience. That experience is provided with increased speed of update for occlusion data by using depth sensor data augmented with lower-level reconstruction data. When operating in real-time dynamic environments, changes in the physical world can be reflected quickly in the occlusion data. Occlusion rendering using live depth data augmented with lower-level 3D reconstruction data, such as a raycast point cloud, can greatly reduce the latency for visual occlusion processing. Generating occlusion data in this way may provide faster operation of an XR system using less computing resources and enabling the system to be packaged in a battery operated wearable device.