Display Rate Control for Video Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In interactive applications such as game screen mirroring, high video latency leads to a poor user experience due to unpredictable waiting times of video frames between decoding and display, which existing technologies fail to adequately address.
Innovation Solution
A method that involves decoding compressed video data, logging decoding and display times to generate a latency history, and adjusting the display rate of the display interface based on this history to optimize waiting times, ensuring they fall within a defined display latency window, thereby reducing video latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If the display rate is increased to reduce waiting times, then video latency is reduced, but frame dropping may occur when waiting times become too short
Solution Approach 1:
The system continuously monitors the waiting time of video frames and uses this feedback to dynamically adjust the display rate. When waiting time exceeds the maximum threshold, the display rate is increased; when it falls below the minimum threshold, the display rate is decreased, creating a closed-loop control system that prevents both excessive latency and frame dropping
Solution Approach 2:
The display rate is transformed from a static parameter to a dynamic one that automatically adjusts based on real-time waiting time conditions. The system enables the display interface to adapt its operating characteristics in response to changing decoding performance and latency requirements
2Reliability
If the display rate is decreased to increase waiting times, then frame dropping is prevented, but video latency increases
Solution Approach 1:
The system uses feedback from waiting time measurements to determine when to decrease the display rate. By monitoring whether waiting times fall below the minimum threshold, the system can proactively reduce the display rate to prevent frame dropping before it occurs, rather than reacting after frames are lost
Solution Approach 2:
The system takes preliminary action by adjusting the display rate in advance based on trends in waiting time. When waiting times approach the minimum threshold, the display rate is decreased beforehand to prevent the condition that would cause frame dropping, maintaining system reliability
3Device complexity
If a fixed display rate is used, then device complexity is reduced, but video latency cannot be optimized for interactive applications
Solution Approach 1:
The system implements self-service by automatically monitoring its own performance and adjusting its display rate without external intervention. The display interface monitors its own waiting times and autonomously modifies its operating parameters, eliminating the need for complex external control mechanisms
Solution Approach 2:
The system changes the display rate parameter dynamically based on performance conditions. By modifying this key operational parameter in response to measured waiting times, the system optimizes video latency without requiring complex hardware architecture
Data Source
AI summary
Aspects of the disclosure provide a method for reducing video latency. The method includes decoding compressed video data into video frames and storing the video frames in a frame buffer for subsequent display by a display interface, logging decoding ending times of video frames and display times of video frames to generate a latency history over a first window of time, and adjusting a display rate of the display interface based on the latency history over the first window of time in order to change waiting times of video frames, the waiting time of a video frame being a difference between the decoding ending time of the video frame and the display time of the video frame.