Adaptive Screen Content Encoding via Motion-Based Layout Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video codec technologies are inflexible in adapting to different motion types in screen content, leading to suboptimal playback quality when reducing temporal or spatial resolution to achieve target bit rates, resulting in either choppy motion or loss of detail.
Innovation Solution
A video encoder system that dynamically determines the current motion type by calculating motion measures in different temporal windows and selectively switches between encoder layouts to adjust frame rate and spatial resolution, using a GPU to offload processing and reduce data transfer, allowing adaptive encoding based on motion characteristics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If screen content is compressed at a lower frame rate to reduce bit rate and computational complexity, then bit rate and computational complexity are reduced, but motion in the reconstructed video content appears choppy
Solution Approach 1:
The encoder dynamically switches between different encoder layouts (first layout with higher frame rate, second layout with lower frame rate) based on the detected motion type of the screen content. This dynamic adaptation allows the system to optimize between bit rate reduction and motion smoothness by selecting the appropriate encoding configuration for each scene.
Solution Approach 2:
The system changes encoding parameters (frame rate, spatial resolution) based on the motion characteristics of the content. By calculating motion measures and comparing them to thresholds, the encoder adjusts parameters to match the content requirements, preventing choppery motion in static scenes while maintaining smooth playback in motion-intensive scenes.
2Productivity
If screen content is compressed at a lower spatial resolution to reduce bit rate and computational complexity, then bit rate and computational complexity are reduced, but the reconstructed video content lacks fine details such as textures
Solution Approach 1:
The encoder dynamically selects between encoder layouts with different spatial resolutions based on motion type detection. This allows the system to preserve fine details in static or low-motion content by using higher spatial resolution encoding, while accepting lower resolution for high-motion content where detail preservation is less critical.
Solution Approach 2:
The system adapts spatial resolution parameters according to the motion characteristics of the screen content. By calculating motion measures and comparing to thresholds, the encoder adjusts spatial resolution to maintain detail quality when appropriate while reducing bit rate when motion justifies lower resolution.
3Manufacturing precision
If manual configuration is used to favor crisp playback by compressing at lower frame rate and higher spatial resolution, then spatial detail is preserved, but motion appears choppy
Solution Approach 1:
The system dynamically adjusts the trade-off between spatial detail preservation and motion smoothness based on detected motion type. Instead of fixed manual configuration, the encoder automatically selects the appropriate encoder layout (prioritizing spatial resolution or frame rate) according to the motion characteristics of each scene.
Solution Approach 2:
The encoder changes encoding parameters (frame rate, spatial resolution) based on motion type detection. This allows the system to automatically favor crisp playback with higher spatial resolution when motion is low, while switching to prioritize motion smoothness with higher frame rates when motion is detected.
4Reliability
If manual configuration is used to favor smooth playback by compressing at higher frame rate and lower spatial resolution, then motion smoothness is improved, but noticeable loss of details occurs
Solution Approach 1:
The system dynamically adapts the balance between motion smoothness and detail preservation by detecting motion type and selecting appropriate encoder layouts. This eliminates the need for fixed manual configuration and allows automatic optimization based on scene characteristics.
Solution Approach 2:
The encoder adjusts spatial resolution and frame rate parameters based on motion type. When high motion is detected, the system favors smooth playback with higher frame rates; when low motion is detected, it switches to preserve details with higher spatial resolution.
5Device complexity
If a single encoder layout is used for all screen content, then device complexity is reduced, but playback quality becomes suboptimal for different motion types
Solution Approach 1:
The system implements dynamic encoder layout selection based on motion type detection. Multiple encoder layouts are prepared in advance, and the system automatically selects the appropriate layout for each scene, providing optimal playback quality without requiring complex real-time configuration changes.
Solution Approach 2:
The system uses pre-configured encoder layouts with different parameter sets (frame rates, spatial resolutions) optimized for different motion types. By detecting motion type and selecting the appropriate layout, the system maintains simple device operation while achieving high playback quality for diverse content.
Data Source
AI summary
Innovations in adaptive encoding of screen content based on motion type are described. For example, a video encoder system receives a current picture of a video sequence. The video encoder system determines a current motion type for the video sequence and, based at least in part on the current motion type, sets one or more encoding parameters. Then, the video encoder system encodes the current picture according to the encoding parameter(s). The innovations can be used in real-time encoding scenarios when encoding screen content for a screen sharing application, desktop conferencing application, or other application. In some cases, the innovations allow a video encoder system to adapt compression to different characteristics of screen content at different times within the same video sequence.