Saliency-Based Environment Modeling for XR Resource Optimization

Solution Overview

Problem

Current methods for modeling physical environments in extended reality systems are resource-intensive, particularly when trying to maintain high resolution and quality across all aspects of a scene, leading to increased computational load and storage requirements, which can detract from user experience by using uniform pixel or polygon densities.

Innovation Solution

The use of saliency values to differentiate modeling features, such as pixel or polygon densities, allows for more computationally intensive features to be applied to salient portions of the environment while using less resource-intensive features for less salient areas, optimizing resource usage and maintaining user experience.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If uniform high resolution modeling features are applied across the entire physical environment, then the quality and detail of the graphical environment is improved, but the computational load and storage requirements increase significantly

Engineering Contradiction:

Improvemodeling precisionVSAvoidcomputational load

Core Design Contradiction:

Manufacturing precisionVSLoss of energy

Solution Approach 1:

The patent applies different modeling features (pixel densities, polygon densities) to different portions of the physical environment based on their saliency values. Salient portions receive higher resolution modeling while less salient portions use lower resolution, resolving the contradiction between overall quality and computational efficiency.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The physical environment is divided into multiple portions with different saliency values, allowing the system to selectively apply different levels of modeling detail to each segment. This segmentation enables quality preservation in important areas while reducing computational burden in less important areas.

Inventive Principle:

Principle #1Segmentation

2Manufacturing precision

If uniform high resolution modeling features are applied across the entire physical environment, then the quality and detail of the graphical environment is improved, but the storage requirements increase significantly

Engineering Contradiction:

Improvemodeling precisionVSAvoidstorage requirements

Core Design Contradiction:

Manufacturing precisionVSQuantity of substance

Solution Approach 1:

The patent applies different modeling features (pixel densities, polygon densities) to different portions of the physical environment based on their saliency values. Salient portions receive higher resolution modeling while less salient portions use lower resolution, resolving the contradiction between overall quality and computational efficiency.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The physical environment is divided into multiple portions with different saliency values, allowing the system to selectively apply different levels of modeling detail to each segment. This segmentation enables quality preservation in important areas while reducing computational burden in less important areas.

Inventive Principle:

Principle #1Segmentation

3Loss of energy

If less resource-intensive modeling features are used for less salient areas, then computing resources and memory are conserved, but the overall quality of the graphical environment may be reduced

Engineering Contradiction:

Improvecomputational loadVSAvoidmodeling precision

Core Design Contradiction:

Loss of energyVSManufacturing precision

Solution Approach 1:

The patent applies different modeling features (pixel densities, polygon densities) to different portions of the physical environment based on their saliency values. Salient portions receive higher resolution modeling while less salient portions use lower resolution, resolving the contradiction between overall quality and computational efficiency.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The system dynamically adjusts modeling features based on real-time saliency assessments. The modeling precision is not static but adapts to the importance of different environment portions, allowing quality to be maintained where needed while optimizing resource usage elsewhere.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS11869144B1Modeling a physical environment based on saliency

Publication Date: 2024.01.09 APPLE INC

AI summary

In some implementations, a device includes one or more sensors, one or more processors and a non-transitory memory. In some implementations, a method includes determining that a first portion of a physical environment is associated with a first saliency value and a second portion of the physical environment is associated with a second saliency value that is different from the first saliency value. In some implementations, the method includes obtaining, via the one or more sensors, environmental data corresponding to the physical environment. In some implementations, the method includes generating, based on the environmental data, a model of the physical environment by modeling the first portion with a first set of modeling features that is a function of the first saliency value and modeling the second portion with a second set of modeling features that is a function of the second saliency value.