Spherical Video Viewport Signaling via SEI Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for signaling a rectangular region of interest in panoramic video systems do not effectively apply to spherical or curved video formats, leading to inefficiencies in displaying the most interesting regions of panoramic video content.
Innovation Solution
A supplemental enhancement information (SEI) message is used to signal metadata that enables the creation of a suggested viewport for panoramic video, allowing for the indication of a most interesting region of interest in spherical representations, which can be applied to various planar projection formats, including equi-rectangular, cube map, and others, enabling legacy encoders and decoders to function without changes to the core encoding or decoding process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing methods for signaling rectangular region of interest are used in panoramic video systems, then the implementation is straightforward for traditional video formats, but the methods do not effectively apply to spherical or curved video formats
Solution Approach 1:
The patent introduces an intermediary coordinate transformation process that converts spherical video coordinates to rectangular projection format. This intermediary step allows existing rectangular ROI signaling methods to work with spherical video content by translating the coordinate systems, thus achieving compatibility without fundamentally changing the core encoding/decoding architecture.
Solution Approach 2:
The patent segments the spherical video processing into distinct stages: coordinate transformation, projection to rectangular format, and then application of standard ROI signaling. This segmentation allows each stage to be handled independently, maintaining simplicity in the core components while adding adaptability through the transformation layer.
2Loss of information
If a suggested viewport is generated using metadata for panoramic video, then the most interesting regions can be prioritized for display, but additional processing overhead is introduced
Solution Approach 1:
The patent performs preliminary coordinate transformation and viewport determination during the encoding phase, storing the results as metadata. This preliminary action shifts the computational burden to the encoding stage, allowing the decoding and playback to proceed more efficiently with pre-computed viewport information already available.
3Adaptability or versatility
If legacy encoders and decoders are used without changes to core encoding or decoding process, then system compatibility is maintained, but the ability to handle spherical video formats with ROI indication is limited
Solution Approach 1:
The patent creates a universal metadata structure that can accommodate both traditional rectangular video and spherical video formats. The same SEI message framework and coordinate transformation procedures can handle multiple video types, allowing legacy encoders and decoders to process spherical video with ROI indication through added functionality rather than fundamental architectural changes.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
A suggested viewport indication is generated and sent for use in a panoramic video. In one example, a method includes receiving encoded video including viewport metadata, decoding the video, extracting the viewport metadata, generating a viewport of the decoded video based on the viewport metadata, and buffering the generated viewport for display.