Hybrid Screen Content Encoding for Compression Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video compression techniques, such as H.264 and HEVC, are inefficient for encoding application screens that combine camera-captured and computer-generated content, as they fail to accommodate the unique properties of screen content, leading to suboptimal compression ratios and visual quality issues.
Innovation Solution
A hybrid approach is employed, where different coding techniques are used for different areas or layers of the screen content, utilizing screen content coding for computer-generated elements and conventional video codecs for natural video layers, enabling efficient partitioning, rendering, and encoding of graphic layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single compression technique (e.g., H.264) is used for all screen content, then the encoding process is simple, but the compression ratio is suboptimal and visual quality deteriorates for certain content types
Solution Approach 1:
The screen content is divided into multiple graphic layers, each representing different content types (e.g., foreground objects, background, text). This segmentation allows different compression techniques to be applied to different layers, optimizing compression efficiency for each content type while maintaining manageable encoding complexity through hierarchical processing.
Solution Approach 2:
Different compression techniques are applied to different graphic layers based on their specific characteristics. For example, lossless or near-lossless compression is applied to text and UI elements requiring sharp edges, while more aggressive compression is applied to background regions. This local quality approach maximizes overall compression efficiency while preserving visual quality where needed.
2Manufacturing precision
If conventional video codecs are used for all screen content, then the encoding process is straightforward, but sharp edges and complex textures are not preserved well
Solution Approach 1:
The patent applies different compression techniques to different graphic layers based on their visual characteristics. Layers containing text, UI elements, or sharp graphical edges use compression methods optimized for preserving these features, while other layers use more efficient compression. This ensures sharp edges and complex textures are preserved where necessary without sacrificing overall compression efficiency.
Solution Approach 2:
By segmenting screen content into multiple graphic layers with different visual characteristics, the system can apply specialized compression techniques to each layer. This segmentation enables better preservation of sharp edges and complex textures in appropriate layers while maintaining high compression efficiency across the entire screen content.
3Productivity
If the entire screen is compressed using a single compression scheme, then the processing is simple, but the compression ratio is not maximized for different content regions
Solution Approach 1:
The screen is divided into multiple graphic layers, each representing different content regions with similar characteristics. This segmentation enables the application of different compression schemes to different layers, maximizing the compression ratio for each content type while keeping processing complexity manageable through systematic layer-based processing.
Solution Approach 2:
The system dynamically selects compression techniques for each graphic layer based on its characteristics and the desired quality-compression tradeoff. This dynamic approach allows the compression ratio to be maximized for different content regions while adapting processing complexity to the specific requirements of each layer, rather than using a fixed single-scheme approach.
Data Source
AI summary
A method for of encoding an application screen comprises partitioning graphic data into a plurality of graphic layers and classifying each of the plurality of graphic layers as either a screen content (SC) or a non-screen content (non-SC) layer. The method further comprises classifying each of the plurality of graphic layers as either a screen content (SC) or a non-screen content (non-SC) layer. Further, the method comprises rendering and encoding the one or more SC layers using a first codec and the one or more non-SC layers using a second codec.