GPU Command Buffer Predication for Low-Latency Frame Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for increasing frame rates in client and cloud gaming, such as reducing render or encode resolution and using neural networks, do not effectively address latency and responsiveness issues in graphics processing.
Innovation Solution
A graphics processor implementation that uses time-based predication with frame generation and scaling via command buffer predication, allowing high-priority commands to bypass lower-priority enqueued workloads for immediate execution, enhancing responsiveness in 3D gaming applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional command buffer execution is used where commands are processed in sequential order, then processing order is maintained and system stability is preserved, but high-priority user-input commands must wait for lower-priority enqueued workloads to complete, increasing latency and reducing responsiveness
Solution Approach 1:
The system dynamically adjusts command buffer processing by introducing priority-based predication that allows high-priority commands to bypass lower-priority workloads. The command buffer execution model transitions from static sequential processing to dynamic priority-driven processing, where the processing order can change based on command priority levels and predication outcomes.
Solution Approach 2:
The system performs preliminary classification of commands into priority levels before execution. High-priority user-input commands are identified and marked in advance, allowing them to be selectively executed before lower-priority workloads complete, rather than waiting for sequential processing to naturally reach them.
2Productivity
If render resolution is reduced to increase frame rate, then more frames can be rendered per second, but image quality and visual fidelity deteriorate
Solution Approach 1:
The system changes the resolution parameter dynamically based on priority. High-priority frames are rendered at full resolution while lower-priority frames can be rendered at reduced resolution. This allows the system to maintain high frame rates through selective resolution reduction rather than uniformly reducing all frames, thus preserving overall image quality while increasing productivity.
3Productivity
If neural networks are used to generate artificial frames for insertion between rendered frames, then frame rate is increased, but processing complexity and computational overhead increase
Solution Approach 1:
Instead of applying neural network frame generation to all frames, the system applies it selectively to specific frames based on priority and timing requirements. This partial application reduces the overall computational overhead and processing complexity while still achieving the frame rate enhancement benefit where most needed.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Described herein is a graphics processor configured to perform time based frame predication to bypass execution of a command buffer based on a comparison with time stamps stored in a time stamp buffer that tracks execution time for command buffers. The graphics processors can bypass a frame that will not complete in time for a target display update and trigger neural frame generation to generate the frame data for the bypassed command buffer. Dynamic render scaling is also described.