360-Degree Video Projection Padding for Boundary Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video codecs struggle to efficiently encode and decode 360-degree videos due to issues with motion field representation and uneven sampling, leading to increased bit usage and viewer discomfort from distorted projections.
Innovation Solution
The use of geometry projections such as equirectangular, cubemap, equal-area, and octahedron projections, combined with interpolation filters and adaptive sampling techniques, to process 360-degree videos, ensuring efficient encoding and decoding while maintaining viewer comfort.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional video codecs are used to encode 360-degree videos, then encoding can be performed with existing tools, but encoding efficiency is poor and bit usage increases
Solution Approach 1:
The patent segments the 360-degree video encoding process by identifying and separately handling boundary blocks that exhibit discontinuous sample values. These boundary blocks are detected based on their location at face boundaries in the projection geometry, and they undergo special processing including padding with derived sample values and selective application of deblocking filters. This segmentation allows the encoder to focus computational resources on problematic areas while maintaining efficiency in regular regions, thereby improving overall encoding efficiency and reducing bit usage.
2Device complexity
If standard projection geometries are used for 360-degree video, then encoding is simplified, but viewer discomfort increases due to distorted projections and uneven sampling
Solution Approach 1:
The patent applies local quality by differentiating processing based on location within the 360-degree video frame. Boundary blocks at face boundaries receive special treatment including padding with derived sample values and selective deblocking filter application, while interior blocks use standard processing. This location-aware approach addresses distortion and uneven sampling artifacts specifically where they occur (at boundaries) without adding complexity to the entire encoding process, thereby reducing viewer discomfort while maintaining encoding simplicity.
3Ease of operation
If boundary blocks with discontinuous sample values are not specially handled, then encoding process is simpler, but encoding efficiency decreases and visual quality deteriorates
Solution Approach 1:
The patent performs preliminary action by detecting boundary blocks and applying padding with derived sample values before the main deblocking filtering process. This pre-processing step ensures that boundary blocks have appropriate sample values to work with, preventing encoding errors and improving efficiency. The preliminary detection and padding operations are integrated into the encoding workflow in a way that maintains process simplicity while significantly improving encoding efficiency and visual quality.
Data Source
AI summary
Processing video data may include capturing the video data with multiple cameras and stitching the video data together to obtain a 360-degree video. A frame-packed picture may be provided based on the captured and stitched video data. A current sample location may be identified in the frame-packed picture. Whether a neighboring sample location is located outside of a content boundary of the frame-packed picture may be determined. When the neighboring sample location is located outside of the content boundary, a padding sample location may be derived based on at least one circular characteristic of the 360-degree video content and the projection geometry. The 360-degree video content may be processed based on the padding sample location.