XR Eye Reprojection Using One Rendered Frame Per Stream
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high computational expense and large bandwidth requirements for rendering and streaming two XR images for each eye in extended reality applications pose a challenge.
Innovation Solution
An XR streaming method that renders a single XR image per frame and employs reprojection techniques to generate images for both eyes, utilizing a predetermined reference position relative to the XR device's momentary position, and optionally includes difference data transmission to correct errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If two XR images are rendered and streamed for each eye, then image quality for both eyes is maintained, but streaming bandwidth requirement increases
Solution Approach 1:
The patent uses a single rendered XR image as a base and creates copies for both eyes through reprojection techniques. Instead of rendering two separate high-quality images, the system renders one master image and generates eye-specific versions by applying reprojection transformations, significantly reducing bandwidth while maintaining visual quality
Solution Approach 2:
The system changes the parameter of image representation by using a single reference image with metadata describing eye position and reprojection parameters, rather than transmitting two complete images. The XR device then adjusts parameters locally to generate the final eye-specific images
2Reliability
If two XR images are rendered for each frame, then realistic 3D experience is achieved, but computational expense increases
Solution Approach 1:
The patent merges the rendering process into a single operation that produces one master XR image for both eyes. The previously separate rendering operations for left and right eyes are combined into one rendering pass, with the XR device performing the eye-specific reprojection operations locally, thereby reducing computational expense at the rendering source
Solution Approach 2:
The XR device performs the reprojection operations itself using its own processing capabilities and the transmitted single reference image. This self-service approach shifts the computational burden from the external rendering system to the XR device, reducing the computational expense of the streaming source while maintaining 3D quality
Data Source
Figure 1~2
Figure 3
AI summary
An extended reality, XR, streaming method of streaming XR images from an XR application instance (26) to an XR device (12) is described. The XR application instance (26) is integrated into a computer device (14) that is provided separately from the XR device (12). The XR streaming method comprises the steps of - rendering, by the XR application instance (26), an XR image to be displayed on the XR device (12) with respect to a predetermined reference position, wherein the predetermined reference position has a predetermined relation to a momentary position of the XR device (12); - streaming, by the XR application instance (26), the rendered XR image to the XR device (12); and - reprojecting, by the XR device (12), the rendered XR image to a first eye position and/or reprojecting, by the XR device (12), the rendered XR image to a second eye position, such that a first XR image for the first eye and a second XR image for the second eye are obtained based on the rendered XR image. Further, an XR projection system (10) is described.