VR-Based AR Experience Authoring Using Depth Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) authoring tools are not designed to run on head-mounted devices and do not allow users to author AR experiences in the same modality as they will be used, limiting the ability to create immersive and efficient AR experiences.
Innovation Solution
The system captures a depth map of a scene using a head-mounted device equipped with a camera and gesture sensor, transmitting it to a virtual reality (VR) processing system, where users can author AR experiences using gestures and voice commands within a fully-immersive VR environment, enabling the alignment and manipulation of digital content with the physical world.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional AR authoring tools are used on general-purpose computers, then the tools have advanced functionality for creating AR experiences, but the authoring process does not match the modality of how AR experiences will be used and requires leaving the AR device
Solution Approach 1:
A virtual reality environment serves as an intermediary between the AR device and traditional authoring tools. The VR environment receives depth maps from the AR device and renders a 3D representation of the scene, allowing authors to interact with the AR scene using natural gestures and voice commands within the VR space, bridging the gap between AR modality and authoring functionality.
Solution Approach 2:
The system creates a digital copy of the real-world scene by generating a depth map using the AR device's camera and gesture sensor. This depth map is then transmitted to the VR processing system which renders a 3D representation, allowing the author to work with a replicated version of the actual scene rather than the physical environment itself.
2Ease of operation
If AR authoring is performed outside the AR device, then traditional input devices can be used for authoring, but the author cannot be fully immersed in the scene while creating the AR experience
Solution Approach 1:
The system transitions the authoring interface from a traditional 2D screen to a 3D virtual reality environment. By rendering the depth map as a 3D representation in VR, the author can interact with the AR scene spatially using gestures and voice commands, achieving full immersion while maintaining authoring productivity through natural interactions.
3Measurement precision
If a head-mounted device with camera and gesture sensor is used to capture the scene, then depth maps can be obtained for VR rendering, but the device must process and transmit data between AR and VR systems
Solution Approach 1:
The system divides the authoring process into separate functional components: the AR device segment captures the scene and generates depth maps with high precision, while the VR system segment handles rendering and authoring interactions. This segmentation allows each component to specialize in its strength while the data transmission pipeline connects them through standardized protocols.
Data Source
AI summary
Examples of techniques for augmented reality authoring using virtual reality are disclosed. In one example implementation according to aspects of the present disclosure, a computer-implemented method includes receiving, by a virtual reality processing system, a depth map from an augmented reality processing system, wherein the depth map is a map of a scene. The method further includes authoring, using the virtual reality processing system, an augmented reality experience in a virtual reality environment based at least in part on the depth map. The method further includes transmitting, by the virtual reality processing system, the augmented reality experience to the augmented reality processing system.