AR Virtual Object Occlusion Rendering for Stable Thin-Object Display
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for displaying and manipulating virtual objects in augmented reality environments are cumbersome, inefficient, and limited, often failing to account for occlusion, sensitivity to thin objects, and displaying instability, while also wasting energy, particularly in battery-operated devices.
Innovation Solution
The system includes a computer system with a display generation component, cameras, and input devices, utilizing methods to determine occlusion and spatial relationships, allowing for efficient display and manipulation of virtual objects by omitting occluded portions, adjusting to thin objects, and providing stable transitions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If conventional methods display all portions of virtual objects, then completeness of virtual object representation is improved, but energy consumption increases and occlusion accuracy deteriorates
Solution Approach 1:
The patent extracts and removes occluded portions of virtual objects from the display output. The system determines which portions of virtual objects are occluded by physical objects and excludes those portions from rendering, thereby reducing energy consumption while maintaining visual accuracy. This is achieved through depth map analysis and occlusion detection algorithms that identify and eliminate unnecessary rendering operations.
Solution Approach 2:
The patent applies partial action by rendering only the visible portions of virtual objects rather than complete objects. By determining occlusion status for different portions of virtual objects and selectively rendering only non-occluded portions, the system reduces computational load and energy consumption while preserving the essential visual information needed for user interaction.
2Loss of information
If conventional methods render all virtual object portions, then visual completeness is improved, but occlusion accuracy deteriorates due to failure to account for physical object boundaries
Solution Approach 1:
The patent replaces traditional geometric occlusion culling methods with a depth map-based approach. By capturing depth information from the physical environment and comparing it with virtual object positions, the system achieves more accurate occlusion determination. This substitution of mechanical/geometric methods with sensor-based depth mapping improves occlusion accuracy while maintaining visual completeness.
Solution Approach 2:
The patent introduces depth maps as an intermediary data structure between the physical environment capture and virtual object rendering. The depth map serves as a mediator that encodes spatial relationships and occlusion information, enabling accurate determination of which virtual object portions should be visible. This intermediary layer resolves the contradiction by providing precise occlusion data without requiring complex geometric calculations.
3Measurement precision
If conventional methods display virtual objects with high sensitivity to thin objects, then detection accuracy is improved, but visual stability deteriorates due to flickering
Solution Approach 1:
The patent adjusts rendering parameters dynamically based on object characteristics. For thin objects, the system modifies detection thresholds and rendering opacity parameters to maintain both detection accuracy and visual stability. By changing parameters such as depth threshold tolerances and blend modes, the system prevents flickering while preserving the ability to detect and render thin objects accurately.
Solution Approach 2:
The patent applies cushioning by introducing temporal smoothing and interpolation for virtual object rendering. Before displaying virtual objects, the system pre-processes depth data and applies smoothing algorithms that prevent abrupt changes in occlusion status. This cushioning effect eliminates flickering by ensuring smooth transitions in rendering parameters, while maintaining sensitivity to thin objects through preserved detection thresholds.
4Loss of information
If conventional methods process complete virtual objects, then rendering completeness is improved, but processing time increases
Solution Approach 1:
The patent segments virtual objects into occluded and non-occluded portions based on depth map analysis. By dividing the rendering task into separate regions, the system processes only the visible portions of virtual objects, significantly reducing processing time. The segmentation is achieved through depth comparison algorithms that partition the virtual object mesh into renderable and non-renderable sections.
Solution Approach 2:
The patent extracts and excludes occluded portions from the rendering pipeline before processing begins. By identifying and removing invisible portions of virtual objects in advance, the system reduces the total processing load and rendering time. This extraction is performed through occlusion culling algorithms that filter out portions of virtual objects that would be hidden by physical objects, maintaining rendering completeness for visible portions only.
Data Source
Figure 1A
Figure 1B
Figure 2A
AI summary
A computer system displays a representation of a camera field of view and receives one or more inputs corresponding to a request to display the representation of the field of view based on a physical object at a first pose, a virtual object at a simulated second pose, and the one or more cameras at a third pose in a physical environment. In response, if a first portion of the virtual object corresponds to physical space that is occluded by the physical object, the system: displays the representation of the physical object; forgoes displaying the first portion of the virtual object; and, if a second portion of the virtual object corresponds to physical space that is not occluded, displays the second portion, including visually deemphasizing a displayed first region of the second portion relative to a displayed second region of the second portion of the virtual object.