Occupancy Grid Scene Understanding for Low-Latency XR Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing XR systems face challenges in accurately rendering virtual content in relation to real objects due to the complexity of human perception and the need for efficient data processing and storage, especially in augmented and mixed reality environments, which often result in latency and inaccurate object placement.
Innovation Solution
The use of occupancy grids to represent the areas occupied by objects, allowing for the generation and transmission of object representations instead of full mesh models, enabling accurate object placement and reducing data storage and latency by performing computations in cloud-based servers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full mesh models are transmitted to XR devices, then rendering accuracy is improved, but data transmission latency and storage requirements increase
Solution Approach 1:
The patent segments the environment model into individual object representations, each containing only the essential data needed for rendering that specific object. Instead of transmitting complete mesh models for all objects, the system transmits segmented object representations with selective geometry data, reducing overall data transmission volume and latency while maintaining placement accuracy.
Solution Approach 2:
The patent extracts and transmits only the critical components needed for object placement and rendering, rather than transmitting complete mesh models. By taking out only the essential geometric data and occupancy information, the system reduces data transmission requirements while preserving the accuracy needed for realistic virtual content placement.
2Measurement precision
If complete object models are stored locally, then rendering quality is improved, but device storage requirements increase
Solution Approach 1:
The patent segments the environment representation into discrete object representations that can be stored and processed independently. Each object representation contains only the necessary data for that specific object, allowing efficient storage management and reducing the total data volume required compared to storing complete mesh models for all objects.
Solution Approach 2:
The patent implements partial action by storing and transmitting only the essential components needed for object rendering and placement, rather than storing complete mesh models. This partial representation approach reduces storage requirements while maintaining sufficient accuracy for realistic virtual content placement in XR environments.
3Measurement precision
If detailed environment models are processed locally, then rendering accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent introduces an intermediary processing stage where a server or cloud system performs complex scene understanding and generates optimized object representations. This intermediary handles the computationally intensive tasks of processing detailed environment models, while the XR device receives pre-processed object representations that require minimal local computation, reducing device complexity while maintaining rendering accuracy.
Data Source
AI summary
This document describes scene understanding for cross reality systems using occupancy grids. In one aspect, a method includes recognizing one or more objects in a model of a physical environment generated using images of the physical environment. For each object, a bounding box is fit around the object. An occupancy grid that includes a multiple cells is generated within the bounding box around the object. A value is assigned to each cell of the occupancy grid based on whether the cell includes a portion of the object. An object representation that includes information describing the occupancy grid for the object is generated. The object representations are sent to one or more devices.