Source Device Frame Capture Buffering for Wireless Display Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Wireless Display (WD) systems face challenges in reducing end-to-end latency and improving video playback quality, as existing technologies often result in delayed and choppy media transmission between source and sink devices.
Innovation Solution
Implementing low latency screen capture and buffering at the source device with minimum-size buffers, using hardware acceleration for processing, and retransmitting encoded frame updates to ensure timely receipt by sink devices, while also prioritizing audio data transport and customizing buffering based on application awareness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If traditional buffering is used in wireless display systems, then video playback quality is maintained, but end-to-end latency increases causing delayed media transmission
Solution Approach 1:
The system dynamically adjusts buffer sizes based on application type and network conditions. For video playback applications, larger buffers are used to ensure smooth playback, while for UI applications, smaller buffers reduce latency. This dynamic adaptation resolves the contradiction by optimizing the buffer size for each specific use case rather than using a fixed buffer size.
Solution Approach 2:
The patent changes the buffer size parameter from a fixed value to a variable that can be adjusted based on application requirements. By modifying this key parameter, the system achieves both low latency for UI applications and high playback quality for video applications, resolving the trade-off between latency and reliability.
2Loss of time
If minimum-size buffers are used to reduce latency, then end-to-end latency decreases, but video playback smoothness deteriorates
Solution Approach 1:
The system implements dynamic buffer size adjustment where the buffer size is not fixed but changes based on the application type. For video playback applications requiring smoothness, larger buffers are allocated. For UI applications requiring low latency, smaller buffers are used. This dynamic behavior resolves the contradiction by adapting to different stability requirements.
Solution Approach 2:
Different buffer sizes are applied to different application types locally. Video playback applications receive larger buffers for smoothness, while UI applications receive smaller buffers for low latency. This localized quality adjustment allows each application type to have optimal buffer characteristics without affecting other applications.
3Productivity
If CPU-based processing is used for frame updates, then processing flexibility is maintained, but processing load increases causing reduced frame rate and increased latency
Solution Approach 1:
The patent replaces CPU-based software processing with hardware-based processing using dedicated video processing units and GPUs. This substitution moves the processing from the general-purpose CPU to specialized hardware, significantly reducing CPU load and increasing frame rates without sacrificing processing capability.
Solution Approach 2:
The system introduces hardware intermediaries (video processing units, GPUs) between the CPU and the display pipeline. These intermediaries handle the computationally intensive video processing tasks, freeing the CPU from heavy processing loads and enabling higher frame rates with reduced energy consumption.
Data Source
AI summary
This disclosure describes techniques to improve a user experience in a Wireless Display (WD) system. The WD system includes a source device that provides media data to one or more sink devices. The techniques are directed toward reducing end-to-end latency in the WD system while improving video playback quality at the sink devices. More specifically, the techniques include low latency screen capture and buffering at the source device. For example, a processing pipeline of the source device may be configured to include minimum-size buffers between processing steps to reduce latency. The techniques include buffering a most recent frame update captured from the media data in the minimum-size buffers and dropping older frame updates when the minimum-size buffers are full. In addition, the processing pipeline may be configured to use hardware acceleration to retrieve the frame updates from the buffers for processing.