XR Image Upscaling for Lower-Bandwidth Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing resolution and image quality in XR devices lead to higher streaming bandwidth requirements, which is computationally expensive and inefficient, especially for battery-powered devices.
Innovation Solution
An XR streaming method that involves rendering XR images at a lower resolution than the device's native resolution, up-converting them at the device side using techniques like pixel duplication or machine-learning, and optionally transmitting difference data to correct errors, thereby reducing bandwidth and computational load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If XR images are streamed at native resolution to maintain image quality, then image quality is improved, but streaming bandwidth and computational load increase
Solution Approach 1:
Instead of streaming high-resolution images and downscaling them at the device (conventional approach), the patent inverts the process by rendering images at lower resolution externally and then upscaling them at the XR device. This reversal of the resolution handling sequence reduces the amount of data that needs to be streamed while maintaining the ability to achieve high display resolution through upscaling techniques.
Solution Approach 2:
The patent uses upscaling techniques (such as pixel duplication or machine learning-based methods) to create a high-resolution copy from a lower-resolution input image. This copying process generates a new high-resolution version of the image at the XR device without requiring the original high-resolution image to be transmitted, thus reducing bandwidth requirements while preserving image quality.
2Measurement precision
If XR images are rendered at higher resolution on external computers, then image quality is improved, but computational cost increases
Solution Approach 1:
The patent inverts the conventional rendering approach by having the external computer render images at lower resolution and then performing upscaling at the XR device. This distribution of computational tasks reduces the computational burden on the external computer while still achieving high display resolution through the upscaling operation performed at the device.
Solution Approach 2:
The XR device performs the upscaling operation itself using its own computational resources, rather than relying on the external computer to provide pre-upscaled images. This self-service approach allows the external computer to focus on rendering tasks while the XR device handles the resolution conversion locally, optimizing the overall computational distribution.
3Quantity of substance
If resolution upscaling is performed at the XR device, then bandwidth is reduced, but image quality may deteriorate due to upconversion errors
Solution Approach 1:
The patent implements a feedback mechanism where the XR device transmits difference data back to the external computer based on the upconverted image quality assessment. This feedback loop allows the system to identify and correct errors introduced during upconversion, thereby maintaining image quality while preserving the bandwidth savings achieved through resolution upscaling.
Solution Approach 2:
The system dynamically adjusts rendering parameters based on the relationship between the upconverted image and the original rendered image. By analyzing differences in image parameters (such as pixel values, gradients, or frequency content), the system can modify rendering settings to compensate for upconversion artifacts and maintain optimal image quality.
Data Source
AI summary
An XR streaming system is disclosed herein that reduces the streaming bandwidth for transmitting XR images based on upscaling techniques. The XR streaming system conserves streaming bandwidth and reduces computational load by streaming an XR image with a first resolution that is smaller than the native or maximum resolution of an XR device. The XR device after receiving the first image, up-converts the resolution of the first XR image to the maximum or native resolution of the XR device. The XR device can up-convert the XR image based on any suitable up-conversion technique.