Graphics Processing Motion Vector Extrapolation for VR Frame Rates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Graphics processing systems for virtual reality (VR) and augmented reality (AR) head-mounted displays face challenges in maintaining high frame rates to prevent motion sickness, as current techniques like frame rate up conversion and bidirectional motion estimation can lead to errors in motion vector identification, resulting in 'holes' and overlap issues in intermediate frames.
Innovation Solution
The system generates extrapolated frames by determining motion vectors from candidate vectors that accurately represent object motion between frames, allowing for increased frame rates without the need for additional rendering, storage, or transmission, thereby reducing processing, memory, and cooling demands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If frame rate up conversion is used to increase display frame rate, then motion sickness is reduced, but processing complexity and power consumption increase
Solution Approach 1:
The patent segments the motion estimation process into two distinct passes: a coarse pass that establishes initial motion vectors, and a fine pass that refines these vectors. This segmentation allows the system to achieve accurate motion compensation without requiring a single overly complex processing stage, thereby reducing overall processing complexity while still enabling high frame rate display to prevent motion sickness.
Solution Approach 2:
The patent performs preliminary motion estimation in a first pass before final frame generation. By pre-calculating motion vectors and storing them for later use, the system prepares necessary data in advance, reducing the computational burden during critical rendering phases and enabling efficient high frame rate operation.
2Productivity
If bidirectional motion estimation is used to generate intermediate frames, then frame rate is increased, but errors in motion vector identification occur causing holes and overlap issues
Solution Approach 1:
The patent divides motion estimation into separate coarse and fine passes, where the coarse pass establishes reliable initial motion vectors and the fine pass refines them. This segmentation prevents the propagation of errors that would occur in a single bidirectional estimation attempt, ensuring accurate motion vectors for all regions including previously problematic holes and overlap areas.
Solution Approach 2:
The patent ensures continuous refinement of motion vectors through multiple passes, maintaining and improving motion vector accuracy throughout the frame generation process. This continuous action eliminates gaps in motion vector coverage, ensuring that all regions of intermediate frames have accurate motion compensation without holes or overlaps.
3Object-affected harmful factors
If high frame rate rendering is implemented to prevent motion sickness, then display quality improves, but power consumption and cooling demands increase
Solution Approach 1:
The patent applies partial action by implementing motion estimation only where necessary - using a coarse pass for initial vectors and a fine pass only for refinement where needed. This selective approach achieves the required frame rate for motion sickness prevention without performing exhaustive processing on all frame regions, thereby reducing power consumption while maintaining display quality.
Solution Approach 2:
The patent discards intermediate computational results from the coarse pass that are not needed for final output, while recovering and reusing accurate motion vectors for frame generation. This efficient resource management reduces unnecessary processing and power consumption while maintaining high frame rate performance to prevent motion sickness.
4Reliability
If multiple passes of motion estimation are performed to ensure complete coverage, then all regions have motion vectors, but processing time increases
Solution Approach 1:
The patent segments motion estimation into a fast coarse pass that covers all regions quickly, followed by a targeted fine pass that refines vectors only where needed. This segmentation ensures complete coverage and reliability of motion vectors while minimizing total processing time by avoiding unnecessary refinement in all regions.
Solution Approach 2:
The coarse pass performs preliminary motion estimation for all regions before the fine pass, establishing a complete baseline coverage. This preliminary action ensures that no regions are left without motion vectors, while the structured two-pass approach optimizes total processing time by preparing data in advance for efficient refinement.
Data Source
AI summary
A graphics processing system includes a processing circuit operable to render or decode a sequence of frames and generate extrapolated frames by extrapolating object motion from rendered or decoded frames. The system also includes a processing circuit operable to extrapolate object motion from first and second rendered or decoded frames in the sequence to a later extrapolated frame. The processing circuit is also operable to test candidate motion vectors from a region of the extrapolated frame through a region of the first frame to a region of the second frame by comparing the region of the first frame with the region of the second frame. A similarity measure from the comparison is used to select a motion vector and an indication representative of the selected motion vector is stored.