Video Encoder Resolution Switching at Random Access Frames
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Frequent resolution changes in video coding increase computational and memory resource consumption, load, and make testing difficult, especially when the resolution of a reference picture differs from that of an inter-predicted picture.
Innovation Solution
An encoder and decoder system that controls resolution changes only at random access pictures, resamples reference images when necessary, and constrains changes based on bitstream monitoring and fixed ratios to reduce resource consumption and testing difficulty.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If resolution changes are allowed frequently in video coding, then adaptability to different picture resolutions is improved, but computational resource consumption and memory resource consumption increase
Solution Approach 1:
The video stream is segmented into random access pictures and non-random-access pictures. Resolution changes are permitted only at random access picture boundaries, creating discrete segments where each segment maintains a consistent resolution. This segmentation approach enables adaptability to different resolutions while preventing frequent changes within segments, thereby reducing computational resource consumption.
Solution Approach 2:
Resolution changes occur periodically at random access picture intervals rather than continuously. The random access pictures serve as periodic checkpoints where resolution transitions are allowed, creating a rhythmic pattern of resolution stability followed by controlled changes. This periodic approach balances adaptability with resource conservation.
2Adaptability or versatility
If resolution changes are allowed frequently in video coding, then adaptability to different picture resolutions is improved, but memory resource consumption increases
Solution Approach 1:
The video coding process is divided into segments separated by random access pictures. Each segment operates with a fixed resolution, allowing the memory buffer to be optimized for that specific resolution. When a resolution change occurs at a random access picture boundary, the system can clear or reset the reference picture buffer, preventing accumulation of memory requirements for multiple resolutions simultaneously.
Solution Approach 2:
Memory buffers are periodically cleared or reset at random access picture boundaries when resolution changes occur. This periodic reset prevents the continuous accumulation of reference pictures in multiple resolutions, thereby controlling memory resource consumption while still allowing adaptability to different resolutions at designated intervals.
3Adaptability or versatility
If resolution changes are allowed frequently in video coding, then adaptability to different picture resolutions is improved, but testing difficulty increases
Solution Approach 1:
The video stream is segmented into fixed-resolution sections bounded by random access pictures. This segmentation creates predictable, manageable units for testing purposes. Test cases can be constructed with known resolution change points at random access picture boundaries, making it easier to detect and measure resolution transitions systematically rather than dealing with arbitrary frequent changes.
Solution Approach 2:
Resolution changes occur at periodic intervals corresponding to random access picture boundaries, creating a regular pattern that simplifies testing. Test systems can anticipate resolution changes at these periodic points and adjust their measurement and detection mechanisms accordingly, reducing testing difficulty compared to irregular frequent changes.
4Use of energy by moving object
If resolution changes are constrained only at random access pictures, then resource consumption is reduced, but resolution flexibility is limited
Solution Approach 1:
The system dynamically adjusts resolution at random access picture boundaries based on content requirements or transmission conditions. While resolution changes are constrained to these specific points, the system maintains flexibility by allowing changes whenever a random access picture occurs, adapting to different scenarios such as scene cuts, quality requirements, or bandwidth variations without requiring changes at every picture.
Data Source
AI summary
An encoder includes: circuitry; and memory coupled to the circuitry. In operation, the circuitry: controls whether to change a resolution of a picture from a resolution of a previous picture preceding the picture in one of display order and encoding order, according to a constraint that allows the changing only when the picture is a random access picture among one or more random access pictures; and when a resolution of a reference picture to be used in encoding of an inter-predicted picture is different from a resolution of the inter-predicted picture, resamples a reference image in the reference picture according to a difference between the resolution of the reference picture and the resolution of the inter-predicted picture, and encodes an image in the inter-predicted picture using the reference image resampled.