Mixed Reality Shared World Model Temporal Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The ability to combine and manage data from multiple mixed reality devices for sharing rich experiences has been limited, making it difficult for users to view, re-experience, or share mixed reality experiences captured by multiple users.
Innovation Solution
A method for presenting a textured shared world model of a physical environment using crowd-sourced geo-located structural and texture data items, which are stitched together to generate a 3D spatial model and filtered temporally for display, allowing users to experience current or past environments through a display device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data from multiple mixed reality devices is combined to create rich shared experiences, then the quality and immersiveness of the shared experience is improved, but the complexity of data management and processing increases
Solution Approach 1:
The system segments the complex task of creating shared experiences into distinct modules: individual devices capture local experiences independently, a server stitches together structural data items to create a shared world model, and separate texture data items are applied to enhance realism. This segmentation allows each component to specialize, reducing overall system complexity while maintaining high experience quality.
Solution Approach 2:
A server acts as an intermediary between multiple mixed reality devices and users. The server receives structural data items from various devices, stitches them together to generate a cohesive 3D spatial shared world model, and manages the complexity of data integration. This intermediary approach allows individual devices to remain relatively simple while achieving complex shared experiences through centralized coordination.
2Manufacturing precision
If texture data items from multiple sources are integrated to enhance realism, then the visual quality of the shared environment is improved, but the processing time and computational resources increase
Solution Approach 1:
The system performs preliminary actions by capturing and storing texture data items (images and videos) from multiple sources in advance, associating them with geographic locations and timestamps. This pre-processing allows the server to quickly retrieve and apply appropriate texture data when generating shared experiences, rather than processing all raw data in real-time, thus reducing processing time while maintaining high visual quality.
3Measurement precision
If temporal filtering is applied to select specific time-stamped data, then the accuracy of re-experiencing past environments is improved, but the complexity of data selection and filtering increases
Solution Approach 1:
The system incorporates feedback mechanisms where users can provide input about temporal filter parameters to refine the selection of texture data items. The server uses this feedback to adjust filtering criteria and select the most appropriate time-stamped data for reconstructing past environments. This feedback loop improves temporal accuracy while distributing the complexity of filtering between automated algorithms and user input.
Data Source
Figure 1
Figure 2~3
Figure 4A
AI summary
Embodiments that relate to presenting a textured shared world model of a physical environment are disclosed. One embodiment includes receiving geo-located crowd-sourced structural data items of the physical environment. The structural data items are stitched together to generate a 3D spatial shared world model. Geo-located crowd-sourced texture data items are also received and include time-stamped images or video. User input of a temporal filter parameter is used to temporally filter the texture data items. The temporally-filtered texture data items are applied to surfaces of the 3D spatial shared world model to generate a textured shared world model of the physical environment. The textured shared world model is then provided for display by a display device.