Adaptive Video Coding for Judder Mitigation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video distribution systems suffer from the 'judder effect' due to lossy compression and frame rate conversion, which degrades the display quality of video content, especially at low frame rates, and fails to effectively mitigate artifacts in all types of video content.
Innovation Solution
The system codes video data as base layer and enhancement layers, where the base layer is encoded at a first frame rate and additional frames are encoded as enhancement layers to mitigate judder artifacts, using frame rate conversion techniques to adaptively add frames based on motion and contrast analysis, and provides metadata for decoder-side processing to optimize frame rate conversion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If frame rate conversion is applied to increase frames per second, then motion smoothness is improved, but processing complexity and artifacts increase
Solution Approach 1:
The video sequence is segmented into regions with different motion characteristics using motion estimation. High-motion regions are identified and processed differently from low-motion regions, allowing selective application of frame rate conversion only where necessary to reduce judder while minimizing overall processing complexity.
Solution Approach 2:
Different processing strategies are applied to different regions of the video based on local motion characteristics. High-motion regions receive enhanced frame rate conversion processing while low-motion regions use simpler processing, optimizing the balance between motion smoothness and processing complexity.
2Speed
If conventional frame rate conversion is applied uniformly to all video content, then motion smoothness is improved, but artifacts are introduced and processing efficiency decreases
Solution Approach 1:
The frame rate conversion process is made dynamic and adaptive rather than uniform. The system dynamically adjusts processing intensity based on real-time analysis of motion characteristics in different video regions, applying higher processing only where needed to maintain efficiency while achieving smooth motion where required.
Solution Approach 2:
Processing parameters such as interpolation complexity and frame generation intensity are changed dynamically based on motion characteristics. High-motion regions trigger more intensive processing parameters while low-motion regions use reduced parameters, improving overall processing efficiency while maintaining motion smoothness where needed.
3Quantity of substance
If lossy compression is used to reduce data size, then bandwidth utilization is improved, but image quality and motion smoothness deteriorate
Solution Approach 1:
Motion estimation and region classification are performed preliminarily during the compression process to identify high-motion regions that will benefit from enhanced processing. This preliminary analysis allows the system to allocate more bits and processing resources to these regions during decoding, improving perceived image quality and motion smoothness without significantly increasing overall data size.
Data Source
AI summary
Judder artifacts are remedied in video coding system by employing frame rate conversion at an encoder. A source video sequence may be coded as base layer coded video at a first frame rate. An encoder may identify a portion of the coded video sequence that likely will exhibit judder effects when decoded. For those portions that likely will exhibit judder effects, video data representing the portion of the source video may be coded at a higher frame rate than a frame rate of the coded base layer data as enhancement layer data. Moreover, an encoder may generate metadata representing “FRC hints”—techniques that a decoder should employ when performing decoder-side frame rate conversion. An encoding terminal may transmit the base layer coded video and either the enhancement layer coded video or the FRC hints to a decoder. Thus, encoder infrastructure may mitigate against judder artifacts that may arise during decoding.