AR Occlusion Rendering for Physical Object Breakthrough

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for interacting with augmented and virtual reality environments are cumbersome, inefficient, and place a significant cognitive burden on users, often requiring multiple inputs and providing insufficient feedback, leading to errors and energy wastage, particularly in battery-operated devices.

Innovation Solution

Implementing computer systems with improved interfaces that utilize touch-sensitive displays, eye-tracking, hand-tracking, and tactile output generators to reduce the number and complexity of user inputs, enhance feedback, and conserve power by reducing visual prominence of virtual content and indicating physical objects in three-dimensional environments.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If conventional methods are used to interact with virtual objects in augmented reality environments, then the system can process and display virtual content, but the interaction becomes cumbersome and requires multiple inputs increasing cognitive burden

Engineering Contradiction:

Improveinteraction efficiencyVSAvoidnumber of inputs required

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The system automatically detects physical objects and determines their properties without requiring explicit user input. The computer system performs self-service by autonomously identifying that a physical object (e.g., a cup) is occluded by virtual content and automatically adjusting the rendering accordingly, eliminating the need for users to manually indicate their intentions through multiple inputs

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system provides real-time feedback by detecting when a physical object is occluded and immediately adjusting the visual prominence of virtual content. This feedback loop allows the system to respond dynamically to the user's interaction context, reducing cognitive burden by automatically adapting to user needs without requiring additional inputs

Inventive Principle:

Principle #23Feedback

2Reliability

If the system provides detailed feedback for user inputs, then interaction accuracy improves, but energy consumption increases particularly in battery-operated devices

Engineering Contradiction:

Improveinteraction accuracyVSAvoidpower consumption

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

Instead of continuously processing and providing feedback on all virtual content, the system periodically updates the visual prominence based on detected physical objects and user interactions. The rendering system adjusts virtual content visibility in response to detected occlusion events rather than maintaining constant high-fidelity rendering, thereby reducing energy consumption while preserving interaction accuracy when needed

Inventive Principle:

Principle #19Periodic action

Solution Approach 2:

The system applies different rendering qualities to different regions of the display based on local conditions. When a physical object is detected to be occluded by virtual content, only the affected local region undergoes rendering adjustments rather than the entire scene, reducing overall energy consumption while maintaining interaction accuracy in the relevant areas

Inventive Principle:

Principle #3Local quality

3Illumination intensity

If virtual content is rendered with high visual prominence, then the user experience is enhanced, but physical objects in the three-dimensional environment become obscured reducing situational awareness

Engineering Contradiction:

Improvevisual prominence of virtual contentVSAvoidvisibility of physical objects

Core Design Contradiction:

Illumination intensityVSLoss of information

Solution Approach 1:

The visual prominence of virtual content is dynamically adjusted based on the detected presence and properties of physical objects. When a physical object is identified as being occluded by virtual content, the system automatically reduces the visual prominence of the virtual content in those regions, creating a dynamic balance that preserves both virtual content visibility and physical object awareness

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system performs preliminary detection of physical objects and their spatial relationships before rendering virtual content. By identifying occlusion relationships in advance, the system can pre-adjust the visual prominence of virtual content to prevent obscuring physical objects, thereby maintaining situational awareness before the user even interacts with the virtual elements

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS12608890B2Obstructed objects in a three-dimensional environment

Publication Date: 2026.04.21 APPLE INC

AI summary

In some embodiments, a computer system reduces a visual prominence of virtual content relative to a three-dimensional environment. In some embodiments, a computer system displays an indication of a physical object in a three-dimensional environment. In some embodiments, a computer system reduces a visual prominence of the virtual content and enables a physical object, such as a person, to break through the virtual content, such as based on a determination that the physical object satisfies one or more criteria to constitute a salient social interaction.