Multi-layer Video Streaming Patch Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video compression techniques face challenges in efficiently encoding and decoding wide-frame/panoramic video sequences, particularly in managing high-resolution data and dynamic viewport changes, which affects bandwidth usage and rendering quality.
Innovation Solution
A method involving a multi-layer transport stream is introduced, where video frames are divided into patches and encoded at different resolutions, with a base layer and supplemental layers, allowing for adaptive processing and rendering based on feedback from rendering devices, enabling efficient transmission and rendering of high-quality panoramic video.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If video frames are encoded at high resolution to maintain quality, then rendering quality is improved, but bandwidth requirements increase
Solution Approach 1:
The video frame is divided into multiple patches, with different resolution levels assigned to different patches. The base layer contains low-resolution data for all patches, while enhancement layers contain high-resolution data only for selected patches. This segmentation allows the system to reduce overall bandwidth requirements by transmitting less data while maintaining high rendering quality for specific regions of interest.
Solution Approach 2:
Different patches within the same video frame are encoded at different resolution levels based on their importance. Patches containing regions of interest (such as objects of interest, facial expressions, or areas indicated by user interaction) are encoded at higher resolution, while other patches use lower resolution. This local quality approach optimizes the trade-off between bandwidth consumption and rendering quality by allocating more data resources to critical areas.
2Manufacturing precision
If all patches are encoded at high resolution, then rendering quality is improved, but encoding complexity and processing time increase
Solution Approach 1:
The encoding process is segmented into two stages: base layer encoding that processes the entire frame at low resolution, and selective enhancement layer encoding that processes only specific patches at high resolution. This segmentation reduces encoding complexity by avoiding the need to encode all patches at high resolution, while still achieving high rendering quality for important regions.
Solution Approach 2:
Instead of encoding all patches at high resolution (excessive action), the system applies high-resolution encoding only to selected patches that contain regions of interest (partial action). This approach achieves sufficient rendering quality for the most important areas while significantly reducing the overall encoding complexity and processing time.
3Manufacturing precision
If the entire video frame is transmitted at high resolution, then rendering quality is improved, but transmission time increases
Solution Approach 1:
The video data is segmented into a base layer and enhancement layers. The base layer is transmitted first and contains sufficient information for acceptable quality rendering. The enhancement layers are transmitted subsequently and provide additional high-resolution data for specific patches. This segmentation reduces transmission time by sending less total data while maintaining the option to achieve high rendering quality when bandwidth is available.
Solution Approach 2:
The base layer is prepared and transmitted in advance, providing immediate rendering capability at acceptable quality. The enhancement layer data is prepared and transmitted subsequently to improve quality for specific regions. This preliminary action approach ensures that rendering can begin sooner with available data, reducing transmission time while preserving the ability to enhance quality.
4Manufacturing precision
If high-resolution encoding is applied to all frames, then rendering quality is improved, but bandwidth usage increases
Solution Approach 1:
The video stream is segmented into a base layer that can be decoded independently at lower bandwidth, and enhancement layers that provide high-resolution details for specific patches. This segmentation allows flexible bandwidth usage where the base layer provides acceptable quality at low bandwidth, while the enhancement layers can be added when more bandwidth is available to achieve high rendering quality for important regions.
Solution Approach 2:
High-resolution encoding is applied locally only to patches containing regions of interest rather than uniformly across the entire frame. This local quality approach reduces overall bandwidth usage by transmitting high-resolution data only where necessary, while maintaining high rendering quality for critical areas such as objects of interest or areas affected by user interaction.
Data Source
AI summary
Systems and methods for enabling a video capture/encoding device to capture frames of a video sequence; obtain feedback from a video decoding/rendering device; divide each frame into an array of patches; select one or more patches of each frame for supplemental processing; cause the frames of the video sequence to be encoded by a video encoding module and cause the selected patches to be processed by one or more supplemental processing module(s); and assemble a multi-layer video transport stream wherein, for each frame of the video sequence, a base layer of the transport stream includes data corresponding to an encoded version of the current frame and one or more supplemental layers of the transport stream include meta-data corresponding to the division of the current frame into patches and data corresponding to the output of the supplemental processing of the selected patches of the current frame.