Rendered Video Slice Display With VSYNC-Aligned Low-Latency Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of high latency and latency instability in cloud gaming due to clock drift between cloud gaming servers and clients, leading to suboptimal synchronization of VSYNC signals and inefficient frame rendering and display.
Innovation Solution
Synchronizing and offsetting VSYNC signals between cloud gaming servers and clients to align their frequencies and adjust timing offsets, allowing for dynamic buffering and overlapping decode and display operations to reduce latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If VSYNC signals are not synchronized between server and client, then clock drift occurs leading to latency instability, but synchronizing VSYNC signals requires complex timing adjustment mechanisms
Solution Approach 1:
The system measures the actual arrival time of video frames at the client and uses this feedback to dynamically adjust the timing offset between server and client VSYNC signals. This closed-loop feedback mechanism compensates for clock drift and network jitter, stabilizing latency without requiring complex predetermined synchronization protocols
Solution Approach 2:
The system dynamically changes the timing offset parameter between server and client VSYNC signals based on measured frame arrival times. By adjusting this timing parameter in response to actual performance data, the system adapts to clock drift and network conditions, achieving latency stability without fixed complex synchronization mechanisms
2Loss of time
If video frames are fully decoded and rendered before display, then display quality is maintained, but one-way latency increases due to sequential processing
Solution Approach 1:
The system performs preliminary decoding of video frames in advance of their display time, allowing the decoding process to overlap with network transmission and client-side processing. This preliminary action reduces the critical path for frame rendering, decreasing one-way latency while maintaining display quality through controlled rendering of decoded frames
Solution Approach 2:
The system enables continuous overlapping of decode, render, and display operations rather than sequential execution. By maintaining continuous useful action across all processing stages and utilizing pipeline parallelism, the system reduces idle time and minimizes one-way latency while ensuring complete frame rendering through synchronized VSYNC signaling
3Productivity
If video frames are transmitted and processed sequentially, then processing accuracy is maintained, but overall processing speed decreases
Solution Approach 1:
The system segments the video frame processing pipeline into distinct parallel stages: network reception, decoding, rendering, and display. Each stage processes frames independently and simultaneously, eliminating sequential bottlenecks while maintaining processing accuracy through stage-specific optimization and synchronized VSYNC coordination
Solution Approach 2:
The system merges multiple processing operations into overlapping parallel execution windows. Network transmission, decoding, and rendering operations are combined and executed concurrently through pipelining, increasing overall frame processing speed while maintaining accuracy through coordinated VSYNC signals that ensure complete processing before display
Data Source
AI summary
A method of cloud gaming is disclosed. The method including receiving an encoded video frame at a client, wherein a server executes an application to generate a rendered video frame which is then encoded at an encoder at the server as the encoded video frame, wherein the encoded video frame includes one or more encoded slices that are compressed. The method including decoding the one or more encoded slices at a decoder of the client to generate one or more decoded slices. The method including rendering the one or more decoded slices for display at the client. The method including begin displaying the one or more decoded slices that are rendered before fully receiving the one or more encoded slices at the client.