Dynamic AR Overlay Display 3D Occlusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) systems face inaccuracies in rendering AR elements due to incomplete or imprecise 3D occlusion, as they fail to accurately consider intervening objects in the environment, leading to unrealistic placements and potential user confusion.
Innovation Solution
The implementation of a system that combines depth maps, object detection, and semantic segmentation to dynamically calculate 3D occlusion, using a masking module and spatial perception engine to refine the occlusion of AR elements based on precise object boundaries and depth information, ensuring accurate masking of obstructed pixels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If existing AR systems place AR elements without considering 3D occlusion, then the placement process is simple and fast, but the realism and accuracy of AR element placement deteriorates
Solution Approach 1:
The system segments the scene into multiple depth layers using depth maps and spatial information. AR elements are placed in specific depth layers corresponding to their real-world positions, allowing accurate occlusion rendering where elements in front of physical objects properly obscure them while elements behind are correctly obscured by those objects.
Solution Approach 2:
The system transitions from 2D image-space AR element placement to 3D spatial placement by incorporating depth maps, spatial segmentation, and occlusion analysis. This dimensional enhancement enables the system to consider the z-axis (depth) when determining AR element visibility and occlusion relationships.
2Manufacturing precision
If existing AR systems use simple AR element placement without occlusion analysis, then the processing speed is fast, but the realism and user experience deteriorates
Solution Approach 1:
The system performs preliminary analysis by generating depth maps, detecting intervening objects, and calculating occlusion relationships before placing AR elements. This pre-computation of spatial relationships enables accurate occlusion rendering without requiring complex real-time calculations during the actual AR element placement and rendering phase.
Solution Approach 2:
The system introduces depth maps and spatial segmentation maps as intermediary data structures that mediate between the raw camera input and the final AR element placement. These intermediaries encode spatial and depth information, allowing the system to efficiently determine occlusion relationships without direct complex ray-tracing or geometric calculations for each AR element.
3Measurement precision
If existing AR systems place AR elements without considering intervening objects, then the implementation is simple, but the accuracy of occlusion rendering deteriorates
Solution Approach 1:
The system replaces complex mechanical or geometric ray-tracing methods for occlusion detection with computational approaches using depth maps and machine learning-based object detection. This substitution enables accurate intervening object detection and occlusion rendering through data-driven methods rather than computationally intensive geometric calculations.
Solution Approach 2:
The system changes the parameter representation from simple 2D image coordinates to multi-dimensional parameters including depth distance, spatial position, and occlusion probability. By representing scene geometry and AR element positions in this enhanced parameter space, the system can accurately determine occlusion relationships through parameter comparison and spatial reasoning.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Three-dimensional occlusion can be used when generating AR display overlays. Depth information can be used to delete portions of an AR element, based on intervening objects between a viewer and the AR element. In cases where the depth information does not impart a complete picture of the intervening objects, additional image processing and object detection systems and techniques can be used to further improve the precision of the occlusion.