Remote Virtual Image Rendering With Late-Stage Reprojection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Remote rendering of virtual images in virtual reality devices introduces latency, compromising the perception of world-locked virtual objects as users move, leading to jittery or swimmy images.

Innovation Solution

A virtual reality device predicts a future pose and requests a virtual image from a remote computer, adjusts the image to match the updated pose using a late stage reprojection mechanism, and displays it at the predicted time to maintain the appearance of world-locked objects.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If remote rendering is used to generate virtual images, then computational load is reduced on the virtual reality device, but latency increases causing image jitter

Engineering Contradiction:

Improvecomputational loadVSAvoidlatency

Core Design Contradiction:

Device complexityVSLoss of time

Solution Approach 1:

The system performs preliminary actions by predicting future device poses and pre-rendering virtual images from the remote computer before they are actually needed for display. This allows the rendering process to start earlier, reducing the effective latency experienced during actual use while maintaining the computational benefits of remote rendering.

Inventive Principle:

Principle #10Preliminary action

2Loss of time

If virtual images are rendered for a future pose, then latency-induced jitter is reduced, but the images must be adjusted after receipt to match the actual pose

Engineering Contradiction:

Improvelatency compensationVSAvoidimage adjustment processing

Core Design Contradiction:

Loss of timeVSDevice complexity

Solution Approach 1:

The system implements feedback by continuously monitoring the actual device pose after rendering and using this information to adjust the pre-rendered virtual images. The adjustment process compares the predicted pose with the actual pose and modifies the image perspective accordingly, ensuring accurate alignment while maintaining the latency compensation benefits.

Inventive Principle:

Principle #23Feedback

3Manufacturing precision

If the virtual reality device waits for remote rendered images, then image quality is maintained, but display smoothness deteriorates due to latency

Engineering Contradiction:

Improveimage qualityVSAvoiddisplay smoothness

Core Design Contradiction:

Manufacturing precisionVSSpeed

Solution Approach 1:

The system performs preliminary rendering actions at the remote computer based on predicted future poses, so that high-quality images are ready before display is needed. This eliminates waiting time during actual display while maintaining image quality, as the rendering work is completed in advance during the prediction window.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system dynamically adjusts the rendering timing and image adjustment parameters based on actual device motion and latency conditions. By making the rendering process adaptive and dynamic rather than static, the system optimizes both image quality and display smoothness according to real-time conditions.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentEP3855290B1Remote rendering for virtual images

Publication Date: 2025.09.03 MICROSOFT TECHNOLOGY LICENSING LLC

Figure 1
Figure 2
Figure 3

AI summary

One or more sensors of a virtual reality device track a pose of the virtual reality device. The virtual reality device requests a virtual image having a perspective corresponding to a future pose from a remote computer. After receiving the requested virtual image, the virtual reality device adjusts the virtual image to an adjusted virtual image having an updated perspective corresponding to an updated tracked pose of the virtual reality device. Then, a virtual reality display displays the adjusted virtual image.