XR Scene Understanding Representation for Generic Virtual Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing XR technologies lack a generic solution for managing the representation of the real part of the scene, limiting the seamless integration of virtual content into the user's environment.
Innovation Solution
A generic API and scene understanding module that initializes and updates the representation of the real scene, integrating it with virtual content, using sensors or databases, and providing a common interface for interaction between modules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a specific solution is used for each engine or XR application, then the solution can be optimized for that specific application, but the overall system complexity increases and there is no generic management approach
Solution Approach 1:
The patent defines a generic scene understanding representation that can be used across different XR applications and engines. The scene description structure with standardized fields (camera pose, trackable poses, 3D representation data) serves as a universal interface that abstracts away the specifics of different XR systems, allowing one representation mechanism to serve multiple applications without increasing overall system complexity
2Productivity
If real-time scene understanding is performed, then seamless integration of virtual content is achieved, but processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary scene understanding actions by capturing and storing scene data (camera poses, trackable poses, 3D representations) before the actual XR rendering occurs. This pre-processing of environmental data allows the system to have ready-to-use scene representations when virtual content needs to be integrated, reducing real-time processing requirements and enabling seamless integration without significant computational delays
3Reliability
If detailed scene representation is maintained, then proper interactions between virtual and real content are ensured, but the data management complexity increases
Solution Approach 1:
The patent segments the scene representation into distinct, manageable components: camera pose data, trackable poses, and 3D representation data. Each component serves a specific purpose and can be independently managed and updated. This segmentation reduces data management complexity by organizing information into logical modules rather than handling it as a monolithic structure, while still maintaining the detailed information needed for accurate virtual-real interactions
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An extended reality system is proposed comprising a presentation engine and a scene understanding module configured to manage representation of the real part of the scene in a generic way. The presentation engine initializes the scene understanding module with the scene description that comprises data describing parameters to integrate the real scene and the virtual part of the extended reality scene. The scene understanding module manages a scene understanding representation thanks to data obtained from sensors or databases and warns the presentation engine of updates of the scene understanding representation. A generic API is defined for interactions between the two modules.