Tile Rectangle Video Coding for Multi-Format Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current viewport-adaptive streaming methods for 360-degree video, such as tile-based encoding and streaming, are constrained by the need for motion-constrained tile sets, which are only available in HEVC, limiting their usability with other coding formats like H.264/AVC, and often require multiple decoder instances, increasing complexity and delay.
Innovation Solution
The method involves splitting a source picture sequence into tile rectangles, encoding each independently, and merging coded tile rectangles vertically to form a coded picture, while managing motion vectors and prediction errors to optimize encoding and decoding efficiency, allowing for the use of different coding formats and reducing decoder instances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If tile-based encoding and streaming is used for viewport-adaptive streaming, then the trade-off between bitrate and quality is improved, but the method is constrained to HEVC format only due to motion-constrained tile sets requirement
Solution Approach 1:
The patent divides the picture into multiple tile rectangles that can be independently encoded and decoded. Each tile rectangle is processed as a separate unit, allowing the system to achieve tile-based streaming benefits without requiring motion-constrained tile sets. This segmentation enables independent handling of different coding formats (HEVC and H.264/AVC) while maintaining the viewport-adaptive streaming capability.
Solution Approach 2:
The patent introduces a new dimension to the encoding approach by vertically merging coded tile rectangles from different coded tile rectangle sequences to form a coded picture. This dimensional change allows the system to bypass the motion-constrained tile set limitation while preserving the independent decodability of tile rectangles, thereby enabling multi-format compatibility.
2Productivity
If multiple decoder instances are used for tile-based encoding, then parallel processing capability is improved, but system complexity and synchronization delay increase
Solution Approach 1:
The patent segments the video stream into independently decodable tile rectangles that can be processed in parallel. By ensuring each tile rectangle contains all necessary information for independent decoding (through the vertical merging approach), the system enables single decoder instance operation while maintaining parallel processing benefits, thereby reducing complexity without sacrificing productivity.
Solution Approach 2:
Each coded tile rectangle is designed to be self-contained and independently decodable, allowing the decoding process to serve itself without requiring complex synchronization between multiple decoder instances. The vertical merging of coded tile rectangles ensures that each unit has the necessary information to be decoded autonomously, eliminating the need for post-decoder synchronization.
3Productivity
If multiple decoder instances are used for tile-based encoding, then parallel processing is enabled, but post-decoder synchronization delay increases
Solution Approach 1:
The patent performs preliminary action by vertically merging coded tile rectangles from different sequences before decoding, ensuring that each coded tile rectangle contains all necessary information for immediate independent decoding. This preliminary structuring eliminates the need for post-decoder synchronization, as each tile rectangle can be decoded and displayed without waiting for other decoders, thereby reducing synchronization delay while maintaining parallel processing capability.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A method comprising splitting a source picture sequence comprising a plurality of tile rectangles into a corresponding number of tile rectangle sequences; encoding each tile rectangle sequence independently into a corresponding number of coded tile rectangle sequences; and merging two or more coded tile rectangle sequences vertically into a coded picture of the bitstream such that each coded tile rectangle forms a coded slice in a coded picture.