Off-Device Computation for High Fidelity AR Recording
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented and virtual reality devices cannot fully record high fidelity experiences due to processing and memory limitations, resulting in low fidelity recordings of computer-generated imagery and digital objects.
Innovation Solution
Off-device computation systems that capture and transmit video and audio data to a server for rerendering and compositing with real-world data, enhancing the fidelity of the recorded experience by processing and rendering at higher resolutions and frame rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If AR/VR devices attempt to record high fidelity experiences locally, then recording quality would improve, but device processing and memory limitations prevent achieving high fidelity
Solution Approach 1:
The patent extracts the high-fidelity rendering computation from the AR/VR device to a remote server. The device captures only essential sensor data and AR object data, while the server performs the computationally intensive rerendering at high fidelity, thereby resolving the contradiction between desired recording quality and device processing limitations.
Solution Approach 2:
The patent introduces a server as an intermediary between the AR/VR device and the final recording output. The server acts as a mediator that receives low-fidelity captured data, rerenders it at high fidelity, and returns the enhanced recording, enabling high-fidelity recording without requiring the device itself to have high processing capabilities.
2Manufacturing precision
If AR/VR devices use local processing for high fidelity recording, then recording quality improves, but device memory and processing power are insufficient
Solution Approach 1:
The patent extracts the memory-intensive rendering operations from the device to the server. By capturing only essential data locally (sensor data, AR object data) and performing the memory-intensive rerendering on the server, the system achieves high-fidelity recording without requiring large device memory capacity.
3Manufacturing precision
If AR/VR devices perform high fidelity rendering locally, then output quality improves, but processing time and computational resources are insufficient
Solution Approach 1:
The patent extracts the computationally intensive high-fidelity rendering process from the device to a remote server with greater computational resources. This allows the device to capture data efficiently while the server performs the resource-intensive rerendering, achieving high rendering quality without bottlenecking the device's processing throughput.
Solution Approach 2:
The patent creates a high-fidelity copy of the AR/VR experience by rerendering the captured scene data on the server. Instead of requiring the device to produce the high-fidelity output directly, the system captures the essential data and generates a high-fidelity copy through server-side processing, thereby achieving high output quality without compromising device productivity.
Data Source
AI summary
Systems and methods are described for recording high fidelity augmented reality or virtual reality experiences through off-device computation. In one implementation, an augmented reality system renders an augmented reality object overlaid over a real-world environment; captures video and audio data of the real-world environment during rendering of the augmented reality object; and stores augmented reality object data associated with the rendered augmented reality object. The augmented reality system transmits the augmented reality object data to a server that rerenders the augmented reality object based on the transmitted augmented reality object data to create rerendered augmented reality object data. The augmented reality system receives the rerendered augmented reality object data or a video of the rerendered augmented reality object data composited with the captured video and audio data.