360 Video Overlay Processing via Metadata Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current VR systems face challenges in providing high-quality 360-degree video content with efficient transmission and accurate reflection of provider intentions, especially in hybrid broadcasting environments, where overlay processing and metadata transmission for 360-degree video are not effectively addressed.
Innovation Solution
The proposed solution involves a method and device for processing and transmitting 360-degree video data, including overlay processing, using ISO-based media file formats and HTTP-based adaptive streaming, which enables efficient transmission and display of user viewpoint-based auxiliary information, and provides a link for efficient screen shifting or auxiliary information provision through signaling information stored and transmitted via SEI messages or VUI.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If overlay processing is implemented for 360-degree video, then user interaction and information display are enhanced, but system complexity and processing overhead increase
Solution Approach 1:
The overlay processing is segmented into distinct functional modules: metadata extraction from bitstream, overlay object identification, spherical surface mapping, and rendering. Each module handles a specific aspect of the overlay process, reducing overall system complexity while maintaining enhanced user interaction capabilities.
Solution Approach 2:
Metadata serves as an intermediary carrier that conveys overlay information from the content source to the rendering system. The SEI message acts as an intermediary format to transmit overlay parameters without requiring direct integration between content delivery and overlay processing systems.
2Loss of information
If metadata transmission for overlay is implemented, then provider intentions are accurately reflected, but transmission bandwidth and data volume increase
Solution Approach 1:
Only essential overlay parameters are extracted and transmitted as metadata, separating critical information (position, size, content identifier) from redundant data. This extraction approach ensures provider intentions are accurately reflected while minimizing transmission overhead.
Solution Approach 2:
Overlay parameters are encoded in compact formats and transmitted as standardized metadata attributes. The use of efficient parameter representation and compression reduces the volume of transmission data while preserving the accuracy of provider intentions.
3Manufacturing precision
If spherical surface rendering is used for overlay, then 360-degree video quality is improved, but rendering complexity and computational requirements increase
Solution Approach 1:
The overlay objects are rendered conforming to the spherical geometry of 360-degree video. By mapping overlay positions onto spherical coordinates and using spherical projection techniques, accurate positioning is achieved while leveraging mathematical models that simplify the rendering calculations compared to general 3D projections.
Solution Approach 2:
Overlay positioning transitions from traditional 2D screen coordinates to 3D spherical coordinates, enabling accurate placement on the curved surface. This dimensional transformation allows precise overlay positioning while using standardized spherical coordinate systems that simplify the underlying mathematical complexity.
4Productivity
If hybrid broadcasting environment is used, then transmission capacity is expanded, but system integration complexity and protocol management increase
Solution Approach 1:
The overlay metadata system is designed to function across multiple transmission protocols and network types (terrestrial and Internet). The standardized metadata format and SEI message structure enable the same overlay processing logic to operate in diverse hybrid broadcasting environments, reducing integration complexity while expanding transmission capacity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A 360 image data processing method performed by a 360 video receiving device, according to the present invention, comprises the steps of: receiving 360 image data; acquiring information and metadata on an encoded picture from the 360 image data; decoding the picture on the basis of the information on the encoded picture; and rendering the decoded picture and an overlay on the basis of the metadata, wherein the metadata includes overlay-related metadata, the overlay is rendered on the basis of the overlay-related metadata, and the overlay-related metadata includes information on a region of the overlay.