Mixed Reality Thumbnail Compositing for Session Continuity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current mixed reality computing devices lack effective methods to maintain continuity and familiarity in the display of three-dimensional virtual models across multiple use sessions, especially when edited and updated, by not adequately incorporating real-world environment data into the rendering process.
Innovation Solution
A mixed reality computing device that includes an image sensor, a display, and a processor, which receives and stores images of the physical environment, renders three-dimensional virtual models, and forms mixed reality thumbnail images by compositing the virtual models with the environmental images, allowing for updating and displaying the models while preserving the original environment's visual characteristics, such as lighting and depth, to maintain continuity and familiarity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If three-dimensional virtual models are updated and re-rendered across multiple use sessions, then the virtual model content becomes more accurate and updated, but the visual consistency and familiarity with the original environment context is lost
Solution Approach 1:
The system segments the thumbnail generation process into two independent components: (1) the real-world environment image captured by the image sensor, and (2) the three-dimensional virtual model rendered from model data. By separating these components, the patent allows the virtual model to be updated independently while preserving the original environment image, thus maintaining visual consistency while improving model accuracy.
Solution Approach 2:
The system performs preliminary capture and storage of the real-world environment image before the virtual model updates. This pre-captured environment image is then reused as the background context in subsequent thumbnail generations, even as the virtual model evolves across multiple sessions. This preliminary action ensures the environment context remains consistent while the model improves.
2Measurement precision
If the system captures and stores real-world environment data for each use session, then the environmental context accuracy improves, but the system complexity and data management burden increase
Solution Approach 1:
The system extracts only the essential real-world environment image data from the complex sensor input and stores it separately from the virtual model data. This extraction approach captures the necessary environmental context (lighting, background, spatial relationships) without storing redundant or excessive data, simplifying data management while maintaining context accuracy.
Solution Approach 2:
Instead of re-capturing and re-processing environment data for each use session, the system creates and reuses a copy of the original environment image. This copying approach preserves the environmental context accurately without requiring complex real-time environmental sensing and processing, thereby reducing system complexity.
3Ease of operation
If the system composites updated virtual models with original environment images, then user familiarity and continuity are maintained, but the rendering and compositing processing time increases
Solution Approach 1:
The system performs partial compositing by combining only the updated virtual model with the pre-captured environment image, rather than re-rendering the entire scene from scratch. This partial action approach maintains user familiarity through environment consistency while minimizing the processing time required for thumbnail generation.
4Adaptability or versatility
If the system maintains separate storage for environment images and virtual model data, then data integrity and flexibility improve, but the storage and processing overhead increases
Solution Approach 1:
The system segments stored data into two distinct categories: environment image data and virtual model data. This segmentation provides data flexibility and adaptability, allowing independent manipulation and updating of each data type while maintaining clear data integrity. The separate storage structure enables versatile data management operations.
Data Source
AI summary
Examples are disclosed herein that relate to the display of mixed reality imagery. One example provides a mixed reality computing device comprising an image sensor, a display device, a storage device comprising instructions, and a processor. The instructions are executable to receive an image of a physical environment, store the image, render a three-dimensional virtual model, form a mixed reality thumbnail image by compositing a view of the three-dimensional virtual model and the image, and display the mixed reality thumbnail image. The instructions are further executable to receive a user input updating the three-dimensional virtual model, render the updated three-dimensional virtual model, update the mixed reality thumbnail image by compositing a view of the updated three-dimensional virtual model and the image of the physical environment, and display the mixed reality thumbnail image including the updated three-dimensional virtual model composited with the image of the physical environment.