HMD Motion Smoothing Using Motion Vectors to Mitigate Judder
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing VR systems using rotation-only re-projection to compensate for missed frames in distributed setups suffer from visual artifacts like judder due to the lack of accounting for moving or animating objects, especially in low-persistence displays.
Innovation Solution
Implement motion smoothing techniques that utilize computer vision algorithms to estimate motion vectors, compressing pixel data on a host computer and transmitting these vectors to a display system like a HMD, which modifies the decompressed pixel data to account for object motion, thereby mitigating judder.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If rotation-only re-projection is used to compensate for missed frames, then frame rate consistency is improved, but visual artifacts like judder occur due to unaccounted object motion
Solution Approach 1:
The patent introduces motion vectors as an intermediary element that bridges the gap between rotation-only re-projection and actual object motion. Motion vectors are extracted from compressed video data and used to adjust the re-projected frames, allowing the system to compensate for both head rotation and object motion independently. This mediator enables the system to maintain frame rate consistency while eliminating judder artifacts.
Solution Approach 2:
The patent segments the motion compensation process into two independent components: head rotation compensation and object motion compensation. By separating these functions, the system can apply rotation-only re-projection for head tracking while simultaneously applying motion vector-based adjustments for object motion, thereby maintaining frame rate consistency without introducing visual artifacts.
2Manufacturing precision
If motion vectors are extracted and applied to re-projected frames, then visual fidelity is improved, but computational complexity increases
Solution Approach 1:
The patent performs motion vector extraction during the video compression phase, before the frames reach the VR display system. By extracting motion vectors in advance during compression, the system avoids the need for complex real-time motion analysis at the display端, thereby maintaining high visual fidelity while minimizing computational complexity in the critical path.
Solution Approach 2:
The compressed video data structure inherently contains motion vector information that serves the dual purpose of compression efficiency and motion compensation. The system leverages this self-contained motion information within the compressed data stream, eliminating the need for separate motion analysis algorithms and reducing overall computational complexity while maintaining visual fidelity.
3Ease of operation
If motion smoothing is implemented in the distributed system, then user comfort is improved, but system resource consumption increases
Solution Approach 1:
The patent extracts and transmits only the essential motion vector data from the host system to the VR display, rather than transmitting complete frame data or complex motion compensation algorithms. By taking out only the necessary motion information, the system achieves motion smoothing that improves user comfort while minimizing the bandwidth and computational resources required in the distributed system.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Described herein are motion smoothing techniques for a display, or display system, such as a head-mounted display (HMD), to account for motion of moving or animating objects in a way that mitigates judder. The display system may be separate from, yet communicatively coupled to, a host computer where a graphics-based application, such as a video game, is outputting frames for rendering on the display system. The host computer may generate motion vectors representing compressed pixel data for transmission to the display system. The motion vectors can be used by the display system to modify pixel data of a frame. The modified pixel data for the frame is "motion-smoothed" for rendering on the display system in a manner that mitigates judder of moving or animating objects.