Split framebuffer encoding for high-resolution remote desktops
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current remote desktop protocols struggle to efficiently stream high-resolution graphical user interfaces to client devices with displays exceeding the resolution limits of native hardware decoders, such as those limited to 1080p, leading to inefficiencies and incomplete utilization of higher resolution displays like 4K.
Innovation Solution
The system divides the framebuffer into multiple child buffers, encodes each child buffer, and transmits encoded child frames to the client device, where they are decoded and combined to reconstruct the original high-resolution frame, using identifiers and padding pixels to ensure compatibility with existing encoders and decoders, and leveraging reference frames for improved efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If a hardware decoder limited to 1080p resolution is used, then power consumption is reduced and decoding is accelerated, but the display resolution capability is limited and cannot utilize high-resolution displays like 4K
Solution Approach 1:
The high-resolution framebuffer is divided into multiple child buffers (e.g., four 1080p buffers for a 4K display). Each child buffer is independently encoded and transmitted to the client device, where hardware decoders can process them at their native 1080p resolution limit. The decoded child frames are then combined to reconstruct the full high-resolution display output, enabling 4K capability while using only 1080p hardware decoders.
2Adaptability or versatility
If the framebuffer is divided into multiple child buffers for encoding, then compatibility with existing hardware decoders is maintained, but the encoding process complexity increases
Solution Approach 1:
The encoding process segments the high-resolution framebuffer into multiple child buffers that match the resolution capabilities of existing hardware decoders. This segmentation enables compatibility with standard 1080p decoders while maintaining a relatively straightforward encoding pipeline that processes each child buffer independently using existing encoding tools and protocols.
Solution Approach 2:
Multiple encoded child frames are combined at the client device to reconstruct the original high-resolution frame. This merging process integrates the outputs of multiple hardware decoders into a single coherent high-resolution display output, effectively combining the capabilities of multiple standard decoders to achieve beyond-standard resolution.
3Adaptability or versatility
If multiple child buffers are transmitted instead of a single frame, then data transmission is compatible with existing protocols, but the quantity of data transmitted increases
Solution Approach 1:
The single high-resolution frame is segmented into multiple lower-resolution child frames for transmission. While this increases the number of transmitted data units, each child frame is smaller in size, allowing existing transmission protocols to handle them efficiently without modification. The total data volume is managed through efficient encoding of each segment.
Solution Approach 2:
Instead of transmitting one large high-resolution frame that exceeds protocol or decoder limits, the system transmits multiple partial frames (child buffers) that collectively represent the complete image. This partial transmission approach ensures compatibility with existing protocols and decoders while achieving the ultimate goal of high-resolution display through aggregation of the partial data sets.
Data Source
AI summary
A virtual desktop can be remoted to a client device by transmitting a video stream using a video coding format, such as H.264, where higher resolution video can be streamed than natively supported by the encoder or decoder. The encoder framebuffer is divided into a plurality of equal size child buffers, each child buffer is assigned an identifier for identifying the child buffer. Padding pixels can be added to the child buffers to make the child buffer size acceptable for the video coding format. The child buffers are encoded using the video coding format and the encoded frames are transmitted to a client. On the decoder side, the encoded child frames are decoded, padding pixels are removed, and the data is used to update corresponding child buffers in a framebuffer on the client display system.