Display Control Unit for Cloud Gaming Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In cloud-based game systems, delays in decoding and displaying image data received from servers can lead to reduced responsiveness and a degraded gaming experience due to network packet delays and jitter.
Innovation Solution
An information processor and method that store image frames and associated metadata, such as input time and transmission frame rate, in memory, allowing the display control unit to select the latest image frame for output when the waiting time exceeds the buffering time, and adjust the display rate in response to changes in transmission frame rate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If image data is transmitted as coded data and stored in a buffer for sequential decoding, then the client can handle network packet delays and jitter, but display delays occur and timely image display cannot be performed
Solution Approach 1:
The patent applies dynamics by making the buffer management adaptive rather than static. The display control unit dynamically adjusts buffer usage based on real-time conditions: when a new image frame arrives, it calculates the elapsed time since the previous frame's display and compares it with a predetermined threshold. This dynamic adjustment allows the system to flexibly respond to varying network conditions, reducing display delays while maintaining reliability during network fluctuations.
Solution Approach 2:
The patent implements preliminary action by pre-calculating and storing metadata (such as arrival timestamps and frame rates) with each image frame before display. This preliminary preparation enables the display control unit to quickly determine whether to display a new frame or wait, eliminating the need for complex real-time decisions during the display process itself and reducing overall display latency.
2Reliability
If a buffer is used to store coded data for sequential decoding, then network jitter is handled, but the sequential process causes delays in reception, decoding, and display
Solution Approach 1:
The patent applies preliminary action by pre-processing and storing essential metadata (arrival timestamps, frame rates) alongside image frames in the buffer. This preliminary preparation allows the display control unit to make immediate display decisions without waiting for sequential decoding completion, thereby maintaining reliability while improving processing speed.
Solution Approach 2:
The patent extracts critical information (elapsed time, frame rate metadata) from the buffered data and uses it for display timing decisions. By separating the display control logic from the sequential decoding process and using extracted metadata, the system achieves parallel operation of decoding and display control, improving overall productivity without sacrificing jitter handling capability.
3Manufacturing precision
If the client waits for complete image data packets, then decoding accuracy is maintained, but responsiveness is reduced and gaming experience is degraded
Solution Approach 1:
The patent applies dynamics by implementing adaptive display timing based on real-time conditions. The display control unit dynamically evaluates the elapsed time since the previous frame and compares it with a threshold, allowing the system to balance decoding accuracy with responsiveness. When time permits, complete frames are decoded accurately; when time is critical, the system adjusts timing to maintain responsiveness while accepting potentially incomplete frames.
Solution Approach 2:
The patent changes the timing parameter dynamically based on network conditions and frame arrival patterns. By adjusting the display timing threshold and using metadata about frame rates and arrival times, the system optimizes the balance between waiting for complete data (accuracy) and displaying frames promptly (responsiveness), thereby improving gaming experience without sacrificing excessive decoding accuracy.
Data Source
Figure 1
Figure 2
Figure 3(A)~3(C)
AI summary
Provided is a device and a method that are capable of display control of image data, which is received through a communication unit, with a small delay. An image frame, and memory input time and transmission frame rate information that serve as metadata are stored in the memory storing the image frame. The display control unit selects the image that is output to the display unit on the basis of the elapsed time from the input time information. The waiting time, which is the elapsed time from the time of input, and the buffering time are compared with each other in each of the queues, and the image frame corresponding to the newest queue, which is a queue among the queues in which the waiting time exceeds the buffering time, is selected as the output image to the display unit. Furthermore, when there is a change in the transmission frame rate, the display rate of the display unit is changed in addition to the change in the transmission frame rate.