Adaptive Screen Coding for Text and Video Blocks

Solution Overview

Problem

Conventional hybrid video coding schemes, such as H.264 and HEVC, are inefficient for encoding and compressing screen data containing text or high-gradient content, leading to unsatisfactory screen sharing performance in applications like remote desktop control and entertainment.

Innovation Solution

An adaptive screen and video coding system that partitions images into blocks, determines the best coding algorithm for each block based on prediction accuracy and efficiency, and applies either a screen coding algorithm for text content or a video coding algorithm, using color classification and index value mapping for efficient compression.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If conventional hybrid video coding schemes (H.264, HEVC) are used for screen data encoding, then the coding is efficient for nature videos, but the coding efficiency deteriorates for text content and high-gradient content which constitute a large portion of screen data

Engineering Contradiction:

Improvecoding efficiencyVSAvoidadaptability to different content types

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The screen data is divided into multiple blocks, and each block is independently analyzed to determine whether it contains text/high-gradient content or natural video content. This segmentation allows different coding schemes to be applied to different regions, resolving the contradiction between optimizing for video content while maintaining performance for text content.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The coding scheme is dynamically selected for each block based on content analysis. The system transitions from a static, uniform coding approach to a dynamic, adaptive approach where the coding method changes according to the local content characteristics, thereby achieving both video efficiency and text content performance.

Inventive Principle:

Principle #15Dynamics

2Ease of operation

If a single coding scheme is used for all screen data, then the implementation is simple, but the screen sharing performance becomes unsatisfactory for diverse content types

Engineering Contradiction:

Improveimplementation simplicityVSAvoidscreen sharing performance

Core Design Contradiction:

Ease of operationVSReliability

Solution Approach 1:

The screen data is divided into multiple blocks, and each block is independently analyzed to determine whether it contains text/high-gradient content or natural video content. This segmentation allows different coding schemes to be applied to different regions, resolving the contradiction between optimizing for video content while maintaining performance for text content.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The coding scheme is dynamically selected for each block based on content analysis. The system transitions from a static, uniform coding approach to a dynamic, adaptive approach where the coding method changes according to the local content characteristics, thereby achieving both video efficiency and text content performance.

Inventive Principle:

Principle #15Dynamics

3Measurement precision

If adaptive coding algorithm selection is implemented for each input block, then the coding accuracy and efficiency improve, but the system complexity increases

Engineering Contradiction:

Improvecoding accuracyVSAvoidsystem complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The screen data is divided into multiple blocks, and each block is independently analyzed to determine whether it contains text/high-gradient content or natural video content. This segmentation allows different coding schemes to be applied to different regions, resolving the contradiction between optimizing for video content while maintaining performance for text content.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The coding scheme is dynamically selected for each block based on content analysis. The system transitions from a static, uniform coding approach to a dynamic, adaptive approach where the coding method changes according to the local content characteristics, thereby achieving both video efficiency and text content performance.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS9699468B2Adaptive screen and video coding scheme

Publication Date: 2017.07.04 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

An adaptive screen and video coding system adaptively selects different coding schemes for coding an image block of a screen or video image based on a type of content included in the image block. The adaptive screen and video coding system further includes a screen coding scheme that classifies pixels of the image block into major colors and escape pixels and convert values of the pixels into index values assigned to the major colors and the escape pixels. Since the number of index values is usually less than a normal range of pixel values, using the index values instead of actual pixel values further facilitate compression and coding of the image block, thus improving coding accuracy and efficiency of the screen or video image. In one embodiment, the adaptive screen and video coding system may be used for screen sharing application.