Auxiliary Frame Video Encoding for Bandwidth Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video encoding methods face challenges in efficiently managing bandwidth, leading to stuttering issues during video streaming due to insufficient internet bandwidth, especially when dealing with high-resolution videos, and fail to effectively adapt to fluctuating bandwidth conditions.
Innovation Solution
The method involves using auxiliary frames in the video coding format, where an auxiliary frame is created with a lower resolution and used to complement the main frame, allowing for dynamic resizing and rate control by scaling down the video data, reducing the bit size of the encoded stream, and utilizing motion vectors to upscale the data to the intended resolution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If video is encoded at high resolution to maintain quality, then video quality is improved, but bandwidth consumption increases causing streaming stutter
Solution Approach 1:
The video encoding is segmented into two distinct components: a low-resolution auxiliary frame containing base layer information, and a high-resolution difference frame containing enhancement information. This segmentation allows the system to transmit essential video content at low bandwidth while providing quality enhancement where bandwidth is available, resolving the contradiction between video quality and bandwidth consumption.
Solution Approach 2:
The system dynamically changes the resolution parameter of the auxiliary frame based on available bandwidth conditions. When bandwidth is limited, the auxiliary frame uses lower resolution, reducing bandwidth consumption. When bandwidth is sufficient, the system can increase auxiliary frame resolution or enhance the difference frame, thereby adapting video quality to bandwidth availability and resolving the quality-bandwidth tradeoff.
2Quantity of substance
If rate control is applied to reduce bandwidth usage, then bandwidth efficiency is improved, but video quality deteriorates at low bitrates
Solution Approach 1:
The auxiliary frame acts as an intermediary that carries essential video information at reduced resolution, while the difference frame serves as a mediator that provides quality enhancement. This intermediary structure allows rate control to reduce bandwidth usage through the auxiliary frame while preserving video quality through the difference frame, even at low bitrates.
Solution Approach 2:
The system adds a temporal dimension to rate control by separating video information into past frames (auxiliary frame) and current frame differences (difference frame). This dimensional separation allows the system to control bitrate by adjusting the allocation between historical information and current details, maintaining quality at low bitrates by prioritizing essential temporal information.
3Length of stationary object
If complex transformations are applied to captured frames to meet resolution requirements, then output resolution is improved, but processing load on video capturing system increases
Solution Approach 1:
The system performs preliminary action by capturing and encoding the auxiliary frame at low resolution before any upscaling or complex transformations are needed. The difference frame is then computed based on this pre-prepared low-resolution reference, significantly reducing the processing load compared to attempting to transform high-resolution captured frames to meet output requirements.
Solution Approach 2:
Instead of the conventional approach of capturing high-resolution frames and then downscaling or transforming them to match output requirements, the system inverts the process by capturing/encoding at low resolution first (auxiliary frame) and then computing differences for enhancement. This inversion dramatically reduces processing load on the video capturing system while still achieving the desired output resolution.
4Quantity of substance
If auxiliary frames are used to reduce bandwidth, then bandwidth requirements are reduced, but encoder complexity increases
Solution Approach 1:
The encoder extracts only the essential low-resolution video information into the auxiliary frame, separating it from the detailed enhancement information in the difference frame. This extraction approach reduces bandwidth requirements by transmitting minimal necessary data while keeping the encoder relatively simple by focusing on extracting rather than processing complex transformations.
Data Source
AI summary
A method and an encoder for encoding a video stream in a video coding format supporting auxiliary frames, where such auxiliary frames, in conjunction with the frames that reference the auxiliary frames, can be used to for rate control, in that the image data of the auxiliary frames comprises a down scaled version of an image data captured by a video capturing device, and that motion vectors of the frame referring to the auxiliary frame are calculated/determined to scale up the down scaled version of the image data to again have the intended resolution.