Screen Map Codec for Variable Content Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing remote screen display systems are inefficient due to inadequate compression methods for variable screen content, leading to high data communication requirements and resource-intensive processing on receiving devices, which consume substantial power.
Innovation Solution
A screen map and progressive standards-based codec system that categorizes screen content into different types, allowing for efficient encoding and offloading of decoding processes from the main CPU to special-purpose hardware, utilizing standards-based codecs like MPEG for image data and text codecs for text data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If traditional MPEG codecs are used for screen content compression, then video content compression is satisfactory, but non-video content (images, text) does not receive substantial data compression
Solution Approach 1:
The screen content is divided into multiple coding units (CU) based on semantic classification (text, image, video, etc.). Each coding unit is then encoded using the most appropriate codec for its specific content type, allowing optimized compression for each segment rather than using a single codec for the entire screen
Solution Approach 2:
The system employs a universal encoder that can handle multiple content types (video, image, text) within the same screen capture. By integrating multiple codec capabilities into a single encoding framework, the system achieves versatile compression that adapts to different content types without requiring separate processing pipelines
2Ease of operation
If substantial processing is performed on the receiving device to decode screen content, then decoding capability is achieved, but power consumption increases substantially
Solution Approach 1:
A standards-based intermediate codec (such as H.264/MPEG-4) is introduced as a mediator between the encoder and the receiving device. This intermediate format leverages existing hardware decoders in receiving devices, eliminating the need for complex custom decoding software and significantly reducing CPU power consumption on the receiving end
Solution Approach 2:
The patent replaces complex software-based decoding mechanisms with hardware-accelerated decoding. By encoding in a standards-based format, the receiving device can utilize dedicated hardware decoders (similar to how video players use hardware for MPEG decoding) to offload computational tasks from the main CPU, thereby reducing power consumption
3Loss of energy
If proprietary compression methods are used for screen sharing, then compression efficiency may be improved, but device complexity increases and hardware support is limited
Solution Approach 1:
The encoding parameters and codec selection are dynamically adjusted based on the detected content type within each coding unit. The system changes encoding parameters (such as quantization levels, transform types, and prediction modes) to match the characteristics of the specific content segment, optimizing compression while maintaining compatibility with standard decoders
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods and systems for delivering screen content to a client device are disclosed. One method includes, for each of a plurality of coding units corresponding to screen regions included in a screen at a particular time, classifying screen content included in the coding unit as having a content type selected from among a plurality of content types, at least one of the content types comprising a screen image type, and, based on a determination that the screen content has a screen image type, applying a progressive standards-based encoding to the screen content of that coding unit. The method also includes transmitting encoded screen content for each of the screen regions to the client device.