Concurrent 3D Canvas Rendering for Mixed Reality Apps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality (AR), virtual reality (VR), and mixed reality (MR) devices are limited in their ability to effectively present two-dimensional (2D) applications in three-dimensional (3D) spaces, lacking adequate solutions for concurrent rendering of multiple apps in a seamless 3D simulation.
Innovation Solution
A system and method that execute multiple applications to generate virtual displays for concurrent rendering in separate canvases within a 3D simulation, using App Space to convert 3D interactions into 2D coordinates, allowing 2D apps to operate within a 3D environment without modification, and managing spatial placement and user interactions across multiple apps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple 2D apps are executed concurrently in a 3D simulation environment, then the versatility and immersive experience are improved, but the device complexity and rendering resource management become more challenging
Solution Approach 1:
The system divides the 3D simulation environment into separate canvases, with each canvas dedicated to rendering a specific 2D app. This segmentation allows independent management of each app's rendering context while maintaining overall 3D spatial coherence, resolving the complexity of concurrent rendering by isolating rendering operations into discrete, manageable units.
Solution Approach 2:
The patent introduces an intermediary layer (the canvas rendering system) that translates between 2D app output and 3D simulation space. This intermediary handles the coordinate transformations, spatial positioning, and concurrent rendering management, shielding the complexity from both the 2D apps and the final 3D display output.
2Adaptability or versatility
If 2D applications are rendered in separate canvases within 3D space, then the immersive experience is improved, but the rendering performance and resource consumption increase
Solution Approach 1:
The system merges multiple 2D canvas renderings into a single unified 3D simulation frame. By combining the rendering operations at the final composition stage rather than maintaining separate full rendering pipelines for each app, the system reduces redundant processing and resource consumption while still achieving concurrent presentation of multiple apps in 3D space.
3Ease of operation
If 3D interactions are converted to 2D coordinates for 2D app compatibility, then the ease of operation is improved, but the measurement precision and interaction accuracy may be affected
Solution Approach 1:
The system adds a spatial dimension layer above the traditional 2D coordinate system. Instead of flattening 3D interactions to 2D, it creates a 3D canvas space where 2D app coordinates are embedded as a subset, allowing precise 3D spatial positioning while maintaining 2D coordinate compatibility for app operations. This dimensional extension preserves precision while enabling compatibility.
Data Source
AI summary
In one aspect, a first device includes at least one processor and storage accessible to the at least one processor. The storage includes instructions executable by the at least one processor to execute a first application (app) to generate a first virtual display for rendering in a first canvas as part of a three-dimensional (3D) simulation. The instructions are also executable to execute a second app to generate a second virtual display for rendering in a second canvas as part of the 3D simulation. The instructions are then executable to concurrently render the first and second canvases at a headset as part of the 3D simulation.