Frame Rendering Extrapolation for Head-Mounted Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional frame rendering technologies require high-end hardware to achieve high frame rates for fluent virtual reality experiences, leading to increased costs and inefficiencies.
Innovation Solution
A frame rendering apparatus and method for head-mounted devices that includes a storage module, motion sensor, and processing module to generate extrapolated frames by performing motion estimation and calibration on consecutive input frames, allowing for high frame-rate display without additional hardware costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional frame rendering technology is used, then hardware cost is reduced, but frame rate is insufficient for fluent virtual reality experience
Solution Approach 1:
The patent applies preliminary action by performing motion estimation and calibration on consecutive input frames before display. The system pre-processes frames to generate extrapolated frames that predict future frame positions, allowing the display to present high-frame-rate content without requiring high-end rendering hardware at the moment of display. This advance preparation resolves the contradiction by shifting computational load from real-time rendering to pre-processing.
Solution Approach 2:
The patent uses copying by generating extrapolated frames that are copies or predictions of future frame states based on motion estimation from previous frames. Instead of rendering each frame independently in real-time, the system creates predicted copies of what future frames will look like, enabling high frame rates through intelligent frame generation rather than brute-force rendering power.
2Reliability
If top-tier hardware setup is used, then fluent virtual reality experience is achieved, but hardware cost increases
Solution Approach 1:
The patent applies parameter changes by transforming the frame rendering approach from direct real-time rendering to a parameter-based prediction system. The system changes the fundamental parameter from rendering every frame at full quality to using motion vectors and calibration data to generate intermediate frames. This parameter transformation maintains fluent experience while dramatically reducing hardware requirements.
Solution Approach 2:
The patent substitutes the mechanical rendering system with a computational prediction system. Instead of relying on raw rendering power to generate each frame, the system uses motion estimation algorithms and calibration data to mathematically predict frame positions and generate extrapolated frames. This substitution replaces brute-force mechanical rendering with intelligent computational methods, reducing hardware cost while maintaining fluency.
3Measurement precision
If motion estimation is performed on all pixel values, then frame extrapolation accuracy is improved, but processing time increases
Solution Approach 1:
The patent applies segmentation by dividing the image into different layers (first image layer and second image layer) and performing motion estimation separately on each layer. This segmentation allows the system to process only relevant pixel values for motion estimation while handling other layers differently, thereby improving extrapolation accuracy for motion-prone regions without unnecessarily processing all pixels, thus balancing accuracy with processing time efficiency.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A frame rendering method used in a head-mounted device that includes the steps outlined below. Input frames are received. First and second orientation information corresponding to a first and a second input frames are retrieved from a motion sensor. Predicted orientation information of a predicted orientation corresponding to a target time spot is generated according to the first and the second orientation information. Orientation calibration is performed on the first and the second input frames according to the first and the second orientation information to respectively generate a first and a second calibrated frames corresponding to the predicted orientation. One of a plurality of extrapolated frames corresponding to the target time spot is generated according to the first calibrated frame and the second calibrated frame.