VDI Client Frame Update Handling for Packet Loss
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing remote desktop protocols face challenges in delivering a high-quality user experience over high latency and high loss wide area networks due to packet loss, which leads to delayed updates and a jumpy user interface.
Innovation Solution
A method that allows a client device to perform frame updates of a desktop GUI without missing image blocks by compressing image blocks into a variable-length stream with positional markers, enabling resynchronization and rendering even with packet loss, and tolerating missing packets to maintain a smooth user experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the remoting protocol re-transmits each lost data packet to ensure reliable transport, then packet loss is resolved and image blocks are complete, but the client device must wait for retransmission before rendering updates, causing delays and reducing update rate
Solution Approach 1:
The image data is divided into multiple independent image blocks that can be processed separately. When packet loss occurs, the client device can render complete image blocks while skipping or interpolating missing blocks, allowing partial frame updates without waiting for retransmission of all lost packets.
Solution Approach 2:
The system performs partial frame updates by rendering only the image blocks that were successfully received, rather than waiting for complete sets. This partial action maintains continuous updates even when some packets are lost, improving the update rate despite incomplete data.
2Manufacturing precision
If the client device waits for lost packets to be retransmitted before performing frame updates, then image completeness is maintained, but the maximum update rate is reduced and user interface appears slow and jumpy
Solution Approach 1:
The system performs frame updates with partial image data by rendering received image blocks and using interpolation or fallback techniques for missing blocks. This allows continuous updates at maximum rate rather than waiting for complete packet sets, maintaining productivity while accepting some degradation in image completeness.
Solution Approach 2:
The client device skips waiting for retransmission of lost packets and proceeds with frame updates using available data. This rushing through the update process maintains high update rates but may result in temporary visual artifacts or incomplete rendering until subsequent packets arrive.
3Reliability
If TCP protocol is used for reliable transport of image blocks, then packet loss is handled reliably, but the protocol overhead and complexity increase compared to UDP
Solution Approach 1:
The patent extracts the reliability function from the transport protocol itself and implements it at the application layer through image block-level error handling. Instead of using TCP's comprehensive reliability mechanisms, the system uses simpler UDP transport with custom logic for detecting and handling lost image blocks, reducing protocol complexity while maintaining sufficient reliability.
Solution Approach 2:
The system introduces an intermediary layer (the image block processing logic) between the transport protocol and the rendering system. This intermediary handles packet loss detection and recovery decisions, allowing the use of simple UDP transport while achieving reliable image block delivery through the intermediary's error handling mechanisms.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In one embodiment, a method displays images from a remote desktop of a desktop GUI on a client device. The method receives a plurality of image blocks for a frame update of an image of the desktop GUI being displayed on the client device. The remote desktop is being run on a host. The client device determines that one or more missing image blocks have not been received for the frame update and determines if the frame update should be performed without the one or more missing image blocks. If the frame update of the desktop GUI should be performed without the one or more missing image blocks, the client device performs the frame update of the desktop GUI using the plurality of image blocks without using the one or more missing image blocks.