Remote Desktop Graphics Prioritization for Low-Latency Display Remoting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current desktop virtualization systems face challenges in delivering high-quality, responsive user experiences for remote access to virtual computing sessions, particularly in environments with high latency and packet loss, such as transoceanic networks, where high-resolution displays and interactive graphics demand efficient graphics remoting and error correction.
Innovation Solution
Implementing a virtualization server that detects interactive graphics regions within a virtual computing session without analyzing the frame buffer, prioritizes them, and sends them over a reliable channel, while sending other content over a lossy channel, using Forward Error Correction (FEC) and User Datagram Protocol (UDP) to enhance interactivity and reduce latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If frame buffer analysis is used to detect interactive graphics regions, then detection accuracy is improved, but processing time and system complexity increase
Solution Approach 1:
The patent extracts only the essential information needed for detection by using virtual session input events (mouse movements, keyboard input, scroll events) rather than analyzing the entire frame buffer. This selective extraction of relevant data reduces processing time while maintaining detection accuracy for interactive regions.
Solution Approach 2:
The system performs preliminary detection of interactive graphics regions by monitoring virtual session input events before actual graphics rendering occurs. This allows the system to pre-identify regions of interest and prioritize their transmission, reducing overall processing time while maintaining accurate detection.
2Reliability
If all frame buffer content is sent with high reliability (FEC), then transmission reliability is improved, but bandwidth consumption and system load increase
Solution Approach 1:
The patent applies different transmission qualities to different regions of the frame buffer based on their importance. Interactive graphics regions detected through virtual session input are sent with high reliability using Forward Error Correction (FEC), while non-interactive regions are transmitted with lower reliability. This localized quality differentiation maintains transmission reliability for critical regions while reducing overall bandwidth consumption.
3Manufacturing precision
If high-resolution displays are used, then display quality is improved, but network latency and bandwidth requirements increase
Solution Approach 1:
The patent segments the high-resolution frame buffer into different priority regions based on virtual session input detection. Only the necessary segments (interactive graphics regions) are transmitted with high fidelity and priority, while other segments are transmitted with lower priority or compressed more aggressively. This segmentation allows high-resolution display quality to be maintained for important regions while reducing overall network latency and bandwidth requirements.
Data Source
AI summary
A computing device may include a memory and a processor cooperating with the memory to host virtual computing sessions to be remotely displayed at a client device via a frame buffer, with the client device being configured to render the virtual computing sessions via a graphical user interface (GUI). The computing device may further detect regions of interactive graphics within a virtual computing session based upon a virtual session input and without analyzing the frame buffer, assign a higher priority to the regions of interactive graphics than other content within the frame buffer, and send the contents of the frame buffer to the client device for rendering in the GUI based upon the assigned priority.