Video Detection in Remote Desktop Protocols Using Tile Heat Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In remote desktop scenarios, it is challenging to determine which data requires encoding to reduce bandwidth requirements, as existing methods lack efficiency in distinguishing between video and visual data updates.
Innovation Solution
A system that includes a video detection module to differentiate between video and visual data by analyzing update frequencies, pixel changes, and heat maps, using encoders like H.264 to encode only video data, thereby optimizing bandwidth usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If data is encoded prior to transmission to reduce bandwidth requirements, then bandwidth consumption is reduced, but it becomes difficult to ascertain which data should be encoded and which does not need encoding
Solution Approach 1:
The display area is divided into multiple tiles, and each tile is further divided into mesh units. This segmentation allows the system to analyze update frequencies at a granular level, identifying which specific regions contain video data that requires encoding versus static visual data that does not need encoding, thereby reducing unnecessary bandwidth consumption.
Solution Approach 2:
Different regions of the display are treated differently based on their update characteristics. By measuring update frequencies for each mesh unit and identifying high-frequency change areas, the system applies encoding only to regions that actually contain video data, while transmitting static visual data without encoding, thus optimizing bandwidth usage.
2Productivity
If video data is identified and encoded separately, then compression ratio improves and bandwidth is reduced, but the complexity of the system increases
Solution Approach 1:
The system segments the display into tiles and mesh units, which simplifies the detection process by breaking down the complex task of identifying video data into smaller, manageable units. This segmentation enables efficient analysis of update frequencies without requiring overly complex processing of the entire display at once.
Solution Approach 2:
The system performs preliminary analysis of update frequencies for each mesh unit before encoding. By pre-identifying which regions are high-frequency change areas, the system can prepare encoding parameters and select appropriate encoding regions in advance, streamlining the overall encoding process and reducing real-time computational complexity.
3Reliability
If all data is encoded, then video quality is maintained, but bandwidth consumption increases unnecessarily
Solution Approach 1:
The system applies different transmission qualities to different regions. High-frequency change areas (video data) are encoded with video codecs to maintain quality, while static visual data is transmitted without encoding. This local differentiation ensures video quality is maintained where needed while avoiding unnecessary bandwidth consumption in static regions.
Solution Approach 2:
Instead of encoding all data, the system applies encoding only partially to the subset of data that actually requires it (video content). This partial action approach maintains sufficient video quality for the encoded regions while avoiding the excessive bandwidth consumption that would result from encoding the entire display output.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Embodiments of the present disclosure provide for dividing a display area having rendered data thereon into a plurality of tiles and determining an update frequency for each tile. A determination is then made as to whether the update frequency of each tile exceeds a threshold. For each tile whose update frequency exceeds the threshold, that tile is marked as a candidate tile. Each of the adjacent candidate tiles are then associated to form a heat map. The heat map is then analyzed to determine whether to encode the rendered data associated with the heat map.