Dynamic Frame Rate Adjustment for Cloud Rendering Jitter

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Cloud-based graphics rendering systems face issues with real-time graphics artifacts such as latency, stutter, and frame tearing due to network jitter, especially when rendering engines cannot synchronize with remote display devices over communication links.

Innovation Solution

A system and method that dynamically adjusts the frame rate of image frame generation based on feedback from the client device, using a jitter buffer with an adaptive size to ensure synchronization with the display device's refresh interval, thereby reducing network jitter effects.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of time

If the rendering engine operates at a fixed high frame rate to reduce latency, then the responsiveness to user input improves, but network jitter causes frame arrival timing deviations leading to stutter and display artifacts

Engineering Contradiction:

ImprovelatencyVSAvoidframe arrival timing consistency

Core Design Contradiction:

Loss of timeVSReliability

Solution Approach 1:

The system dynamically adjusts the frame rate of the rendering engine based on real-time network conditions and frame arrival timing feedback. Instead of operating at a fixed frame rate, the rendering engine adapts its output rate to compensate for network jitter, ensuring frames arrive at the display device at the correct refresh intervals without causing stutter or tearing artifacts

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The display device monitors the actual arrival times of transmitted frames and sends timing feedback to the rendering engine. This feedback loop enables the rendering engine to adjust its frame generation rate dynamically, correcting timing deviations caused by network jitter and maintaining synchronized frame delivery to the display

Inventive Principle:

Principle #23Feedback

2Reliability

If a large jitter buffer is used to absorb network timing variations, then frame arrival timing consistency improves, but display latency increases

Engineering Contradiction:

Improveframe arrival timing consistencyVSAvoiddisplay latency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The jitter buffer size is dynamically adjusted based on current network conditions and frame arrival patterns. When network jitter is low, the buffer size is reduced to minimize latency. When jitter increases, the buffer size expands to absorb timing variations. This dynamic adaptation allows the system to maintain reliable frame timing while minimizing unnecessary latency

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes the jitter buffer size parameter in response to varying network conditions and frame arrival timing deviations. By adjusting this parameter dynamically rather than using a fixed buffer size, the system optimizes the trade-off between timing consistency and latency based on real-time operational conditions

Inventive Principle:

Principle #35Parameter changes

3Loss of time

If the rendering engine generates frames faster than the display refresh rate to compensate for network delay, then latency is reduced, but frame synchronization with display refresh cycles deteriorates causing tearing artifacts

Engineering Contradiction:

Improvedisplay latencyVSAvoidframe timing precision

Core Design Contradiction:

Loss of timeVSManufacturing precision

Solution Approach 1:

The display device provides feedback about its actual refresh cycle timing to the rendering engine. Using this feedback, the rendering engine synchronizes its frame generation to match the display's refresh rate, ensuring frames arrive precisely when needed for display. This prevents screen tearing caused by unsynchronized frame delivery while maintaining low latency through optimized timing

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system prepares frames in advance with timing information embedded, allowing the display device to schedule frame presentation at the optimal moment aligned with its refresh cycle. This preliminary timing preparation enables precise synchronization between frame arrival and display refresh, eliminating tearing artifacts

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS10741143B2Dynamic jitter and latency-tolerant rendering

Publication Date: 2020.08.11 NVIDIA CORP

AI summary

Systems and techniques for streaming video with dynamic jitter tolerance are described. In one example, a system includes a server executing an application and generating image frames associated with the application at a frame rate, and a client which displays the image frames on a display that has a predetermined refresh rate and which monitors arrival times of the image frames in relation to the predetermined refresh rate. The server is further configured to dynamically change the frame rate based on the monitoring so that the frame rate more closely corresponds to the predetermined refresh rate of the client's display.