Omnidirectional Video Encoding via Resolution-Adaptive Sub-Picture Partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding systems face challenges in efficiently encoding and decoding omnidirectional video content, particularly in adapting to different viewport resolutions and bitrates, which affects the quality and efficiency of streaming omnidirectional video.
Innovation Solution
The method involves resolution-adaptive tile merging for viewport-adaptive streaming, where input picture sequences are encoded into motion-constrained coded sub-picture sequences using different sub-picture partitioning grids, and these sequences are encapsulated in a container file format, allowing for selective decoding based on viewport requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If omnidirectional video is encoded at high resolution to maintain video quality, then video quality is improved, but bitrate requirements increase
Solution Approach 1:
The omnidirectional video is divided into multiple sub-picture sequences using different sub-picture partitioning grids. Each sub-picture sequence covers a specific portion of the spherical field of view and can be independently encoded at different resolutions, allowing the system to provide high quality where needed while reducing overall bitrate
Solution Approach 2:
Different sub-picture sequences are encoded at different spatial resolutions based on their importance and the expected viewport coverage. High-resolution encoding is applied to regions more likely to be in the viewport, while lower resolution is used for peripheral regions, optimizing the balance between quality and bitrate
2Adaptability or versatility
If multiple resolution versions are encoded to support adaptive streaming, then adaptability is improved, but device complexity increases
Solution Approach 1:
The video is segmented into multiple sub-picture sequences with different partitioning grids, where each sequence can be independently decoded at appropriate resolutions. This segmentation allows the decoder to select and combine only the necessary sub-sequences for the current viewport, reducing decoding complexity compared to processing multiple complete resolution versions
Solution Approach 2:
The same sub-picture sequences can serve multiple resolution requirements through selective decoding and combination. The motion-constrained coded sub-picture sequences are designed to be universally applicable across different viewport sizes and resolutions, eliminating the need for separate encoding of multiple complete resolution versions
3Productivity
If motion-constrained coded sub-picture sequences are used for viewport-adaptive streaming, then streaming efficiency is improved, but encoding complexity increases
Solution Approach 1:
The video is pre-encoded into motion-constrained coded sub-picture sequences with different sub-picture partitioning grids before streaming. This preliminary action prepares the video data in a format that enables efficient viewport-adaptive decoding at the client side, where only the necessary portions need to be processed based on the current viewport, improving streaming efficiency while distributing encoding complexity to the server side
Data Source
Figure 1a~1b
Figure 2a~2b
Figure 3~4
AI summary
There are disclosed various methods, apparatuses and computer program products for video encoding and decoding. In some embodiments a first input picture sequence comprising pictures of an equirectangular projection format is obtained. The first input picture sequence is encoded at a first spatial size characterized by a first number of pixels into a first set of motion-constrained coded sub-picture sequences using a first sub-picture partitioning grid with a first sub-picture width in pixels. The first input picture sequence is also encoded at a second spatial size characterized by a second number of pixels into a second set of motion-constrained coded sub-picture sequences using a second sub-picture partitioning grid with the first sub-picture width in pixels, wherein the second number of pixels is an integer fraction of the first number of pixels. The pictures of the first input picture sequence are shifted horizontally in a wraparound manner; and the horizontally shifted first input picture sequence is encoded at the second spatial size into a third set of motion-constrained coded sub-picture sequences using the second sub-picture partitioning grid.