Sub-frame Scanout for VR Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graphics processing systems for head-mounted displays face significant rendering latency issues, particularly in virtual reality applications, where high refresh rates are required to prevent user nausea and lag, due to the computational intensity of rendering frames and the disconnect between visual and motion stimuli.
Innovation Solution
The system divides each frame into multiple sub-frames, allowing for sequential rendering and scanning out of each slice independently, with sensor data from head-mounted display sensors used to update parameters for each slice, thereby reducing latency by overlapping processing and updating of sub-frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If the refresh rate is increased to reduce motion sickness and lag in virtual reality, then the responsiveness and user comfort improve, but the rendering latency increases because the system cannot generate frames fast enough
Solution Approach 1:
The patent divides each frame into multiple slices (e.g., 4 slices per frame) and renders them sequentially rather than waiting to complete the entire frame. This segmentation allows the display to show partial frame data sooner, effectively reducing the perceived latency while maintaining the high refresh rate required for VR comfort.
2Manufacturing precision
If the frame complexity is increased to improve visual quality, then the image fidelity improves, but the frame rate decreases making it impossible to meet the 30 fps minimum requirement
Solution Approach 1:
The patent applies partial action by rendering only portions of the frame (slices) rather than complete frames. This allows the system to process and display visual data at a higher effective rate, maintaining image quality for rendered portions while achieving the necessary frame rate throughput that would be impossible with complete frame rendering.
3Stability of the object's composition
If the system waits to complete full frame rendering before display, then the image completeness is maintained, but the latency becomes unacceptable for virtual reality applications
Solution Approach 1:
The patent performs preliminary action by rendering and preparing slice data in advance before the complete frame is ready. Each slice is rendered independently and made available for display sooner, allowing the system to show partial visual information earlier rather than waiting for complete frame assembly, thus reducing latency while maintaining acceptable visual completeness.
Data Source
AI summary
A system, computer readable medium, and method for sub-frame scan-out are disclosed. The method includes the steps of dividing a frame into a plurality of slices. For each slice in the plurality of slices, the steps further include sampling a sensor associated with a head mounted display to generate sample data corresponding to the slice; adjusting one or more parameters associated with rendering operations for the slice based on the sample data; and rendering primitive data associated with a model according to the rendering operations to generate image data for the slice. Each slice is a portion of the frame and corresponds to different sample data from the sensor. Thus, adjusting of the parameters is different for each slice of the frame.