Video Encoder Pre-processing Border Region Resolution Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video call systems face challenges in dynamically adapting video resolution to changing channel conditions and processing resources without requiring frequent changes to the encoder's frame size, which can impact bitrate and processing efficiency, and not all encoders or decoders support variable frame sizes.
Innovation Solution
The approach involves cropping the actual video content to a smaller size than the encoder's frame and adding a redundant border, allowing the effective resolution to be reduced independently of the encoder's frame size, with the border incurring minimal bits during encoding, thus enabling smoother resolution changes without altering the encoder's frame size.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the encoder frame size is changed to adapt resolution to channel conditions and processing resources, then the video resolution can be adapted to network capacity and processing power, but frequent resolution changes impact bitrate efficiency and processing resource usage
Solution Approach 1:
The frame is segmented into an active video content region and a border region. The border region is excluded from encoding operations, allowing the active region to be dynamically resized without changing the overall encoder frame size. This segmentation enables resolution adaptation while maintaining encoding efficiency at the original frame size.
Solution Approach 2:
The solution moves the resolution adaptation problem from the encoding dimension to the preprocessing dimension. By cropping the active video content region before encoding and adding a border region, the effective resolution is changed without altering the encoder's operating frame size, thus avoiding the inefficiencies of frequent encoder reconfiguration.
2Adaptability or versatility
If the encoder frame size is changed to adapt resolution, then the video resolution can be adjusted, but not all encoders or decoders support variable frame sizes
Solution Approach 1:
The border region serves multiple functions: it enables resolution adaptation, maintains compatibility with fixed-frame-size encoders and decoders, and provides a mechanism for smooth resolution transitions. By keeping the overall frame size constant while varying the active content region, the solution works with any standard encoder/decoder without requiring variable frame size support.
3Manufacturing precision
If the video resolution is increased to improve quality, then the receiving user perceives better quality, but higher resolution incurs higher bitrate and more processing resources
Solution Approach 1:
The solution applies different processing to different regions of the frame. The active video content region is encoded at the desired quality level, while the border region is excluded from encoding entirely. This allows quality optimization for the important content area without incurring the bitrate cost of encoding the entire frame at high resolution.
Data Source
Figure 1~3
Figure 4~6
AI summary
A user terminal for participating in video calls comprises: an encoder having a frame size, being the size in pixels at which it encodes frames of video; and a pre-processing stage which supplies a sequence of frames to the encoder at that frame size, each frame comprising at least an image region representing a source video image at a respective moment in time. The pre-processing stage is configured to supply at least some of the frames to the encoder in a modified form, by resizing the source video image to produce the image region of each modified frame with a size smaller than the frame size of the encoder, and combining with a border region such that the modified frame matches the frame size of the encoder. The encoder encodes the frames at the frame size and transmits them to a receiving terminal as part of a live video call.