Server Frame Rendering Timing Adjustment for Cloud Gaming Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Frame latency in cloud gaming systems is exacerbated by VSYNC timing at the client device, which introduces additional delay between when a decoded frame is available and when it is presented, despite efforts to reduce latency in other pipeline stages.
Innovation Solution
Implementing dynamic frame start shifting in the server's streaming pipeline, where the client device observes VSYNC presentation latencies and feeds this information back to the server, allowing the server to adjust the frame start delay to align decoded frame availability with VSYNC signal assertions, thereby reducing VSYNC present latency and overall frame latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If VSYNC timing is enabled at the client device to mitigate tearing and visual artifacts, then display stability is improved, but frame latency increases due to the delay between decoded frame availability and VSYNC signal assertion
Solution Approach 1:
The server device performs preliminary actions by adjusting the frame rendering timing to anticipate when decoded frames will be available at the client device. The server calculates the time required for encoding and transmission, and proactively schedules frame rendering to complete just in time for VSYNC assertion, thereby reducing the waiting period without compromising display stability
Solution Approach 2:
The system implements feedback by having the client device measure actual VSYNC present latency and communicate this information back to the server device. The server uses this feedback to dynamically adjust its frame rendering timing, creating a closed-loop control system that continuously optimizes frame latency while maintaining VSYNC synchronization for display stability
2Productivity
If encoding or decoding efficiency is improved to reduce latency in those stages, then processing speed is improved, but overall frame latency remains unchanged because VSYNC assertion timing is fixed
Solution Approach 1:
The invention introduces dynamics by making the frame rendering timing adjustable rather than fixed. The server device dynamically modifies when frame rendering starts and completes based on measured VSYNC latency, transforming a static processing pipeline into an adaptive system that can optimize overall frame latency regardless of fixed VSYNC timing at the client device
Data Source
AI summary
In a cloud gaming system or other remote video streaming system, a client device and a server coordinate to introduce an adjustable delay in the frame start timing in the frame rendering pipeline at the server to reducing vertical synchronization (VSYNC) presentation latency, and thus reduce overall frame latency. In implementations, the coordination between the client device and the server includes the client device observing the current VSYNC presentation latencies in recently processed video frames reporting this observed VSYNC presentation latency to the server. The server uses this feedback to determine a frame start delay that is then used to introduce a frame start shift for an upcoming frame and subsequent frames, thereby shifting the server rendering and encoding pipeline back so that the resulting video frames are made available to present at the client device closer to their respective VSYNC signal assertions.