Omnidirectional Video 3D Object Overlay Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies for omnidirectional video coding, such as MPEG Omnidirectional Media Format (OMAF), lack support for incorporating three-dimensional (3D) objects as overlays, which are better suited for six degrees of freedom (6DoF) capabilities but are not integrated into OMAF content representation.
Innovation Solution
A method and apparatus that generate and encode a bitstream for omnidirectional visual media content, including information about a 3D object as an overlay, and signal the relationship between the coordinate systems of the omnidirectional visual media content and the 3D object, enabling the inclusion and proper alignment of 3D objects within the omnidirectional media.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If 2D rectilinear content is used for overlays in OMAF, then the content can be rendered with current technology, but the 6DoF capabilities and spatial alignment with omnidirectional content are limited
Solution Approach 1:
The patent introduces coordinate system transformation as an intermediary mechanism that bridges the 3D object's native coordinate system and the omnidirectional content's coordinate system. This transformation layer enables 3D objects to be properly aligned and integrated into the omnidirectional environment without requiring fundamental changes to either the 3D object representation or the OMAF framework
Solution Approach 2:
The patent changes the parameter representation by encoding coordinate system relationship information (such as transformation matrices, rotation angles, or position offsets) into the bitstream. This allows the rendering system to dynamically adjust the 3D object's parameters (position, orientation, scale) based on the viewer's perspective and head motion, enabling true 6DoF interaction
2Adaptability or versatility
If V-PCC or 3D objects are incorporated into OMAF content representation, then better 6DoF overlay capabilities are achieved, but the current OMAF specification lacks support for such integration
Solution Approach 1:
The patent segments the overlay content into distinct 3D objects that can be independently encoded, transformed, and rendered. Each 3D object is treated as a separate entity with its own coordinate system and transformation parameters, allowing flexible integration into the omnidirectional content without compromising the overall OMAF structure
Solution Approach 2:
The patent transitions from 2D rectilinear overlay representation to 3D volumetric object representation. This dimensional change enables the overlays to occupy and interact with three-dimensional space, providing true 6DoF capabilities where objects can be viewed from any angle and position within the omnidirectional environment
3Manufacturing precision
If coordinate system relationship signaling is added to the bitstream, then proper alignment and integration of 3D objects are enabled, but the bitstream complexity increases
Solution Approach 1:
The patent applies partial action by signaling only the essential coordinate system relationship parameters needed for proper alignment, rather than transmitting complete transformation matrices or full coordinate system definitions. This selective signaling approach provides sufficient precision for spatial alignment while minimizing the additional bitstream overhead
Data Source
Figure 1
Figure 2a
Figure 2b
AI summary
A method comprising: generating a bitstream defining a presentation, the presentation comprising an omnidirectional visual media content; encoding, into the bitstream, information about a three-dimensional (3D) object to be included as an overlay into said omnidirectional visual media content; and signaling, in or along the bitstream, a relationship between a coordinate system of the omnidirectional visual media content and a coordinate system of the 3D object.