Virtual Channel Prioritization for Low-Latency Remote Desktop Input
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current desktop virtualization systems face challenges in delivering high-quality, low-latency remote access to virtual computing sessions, especially in high-latency and packet-loss network conditions, leading to laggy user experiences with tasks like typing, scrolling, and interacting with complex graphics, particularly in cloud-based environments.
Innovation Solution
The system prioritizes interactive graphics regions within a virtual computing session by detecting user and system hints without analyzing the frame buffer, sending them over a reliable channel, and using forward error correction (FEC) to enhance rendering quality and speed, while sending less critical content over a lossy channel, thereby optimizing bandwidth and reducing retransmission delays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If all frame buffer content is sent over a reliable channel with forward error correction, then rendering quality and fidelity are improved, but bandwidth consumption increases and latency worsens
Solution Approach 1:
The patent segments the frame buffer content into multiple priority levels (high-priority interactive regions and low-priority non-interactive regions). High-priority regions containing interactive graphics are sent over reliable channels with forward error correction, while low-priority regions are sent over lossy channels without FEC, thereby reducing overall bandwidth consumption and latency while maintaining rendering quality for critical content.
Solution Approach 2:
The patent applies different transmission quality levels to different regions of the frame buffer based on their interactive importance. High-priority regions receive full error protection and high-quality transmission, while low-priority regions use compressed lossy transmission, optimizing the balance between rendering quality and transmission efficiency for each local region.
2Reliability
If forward error correction is applied to all transmitted data, then packet loss resilience is improved, but bandwidth consumption and processing overhead increase
Solution Approach 1:
The patent divides transmitted data into high-priority and low-priority segments. Forward error correction is applied only to high-priority data containing interactive graphics elements that require reliable delivery, while low-priority data is transmitted without FEC, reducing bandwidth consumption and processing overhead while maintaining packet loss resilience for critical content.
3Measurement precision
If the system analyzes the frame buffer to identify interactive regions, then prioritization accuracy is improved, but processing time and computational overhead increase
Solution Approach 1:
The patent performs prioritization analysis at the application layer before frame buffer rendering, identifying interactive regions in advance based on application metadata and user input patterns. This preliminary action allows the system to pre-tag regions with priority information, enabling efficient prioritized transmission without requiring time-consuming frame buffer analysis during the rendering pipeline.
4Ease of operation
If high-priority interactive regions are detected and prioritized, then user interactivity is improved, but system complexity increases
Solution Approach 1:
The patent introduces an intermediary prioritization module that sits between the application layer and the rendering pipeline. This module intercepts application metadata and user input events, identifies interactive regions, and tags them with priority information before rendering. The tagged frame buffer data then flows through the existing rendering and transmission pipelines with minimal additional complexity, as the prioritization logic is encapsulated in the intermediary module.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In a virtual desktop infrastructure method for improving transfer protocol for remote computing, comprising sending traffic over multiple virtual channels having different priorities and packet loss rates. Applied error correction (FEC) may depend on the robustness of the network. User events are classified and assigned to a virtual channel based on the classification. High priority events include keyboard events touch up touch down events, mouse up mouse down, pen up pen down. Low priority events include touch move, mouse drag, mouse wheel pen drag and pen move events.