Backward Compatible Immersive Video Stream Formatting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There is a lack of a format for a backward compatible immersive video stream that can be properly displayed by both legacy and immersive video rendering devices, as legacy devices struggle to render immersive videos due to their inability to map on three-dimensional convex surfaces, and immersive devices fail to display legacy videos correctly.
Innovation Solution
A method and device are proposed to generate a backward compatible video stream by capturing and removing parts of an immersive video to create a legacy video, which is then packaged with the immersive video, allowing legacy devices to render the legacy video and immersive devices to map back the legacy video onto the original immersive video surface.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If immersive video is rendered on legacy devices using flat rectangular mapping, then the video can be displayed on legacy devices, but the immersive experience is lost and distortion occurs
Solution Approach 1:
The immersive video stream is segmented into multiple components: a base layer containing the full immersive video and enhancement layers containing rectified video portions. This segmentation allows legacy devices to use only the base layer while immersive devices can utilize both layers for enhanced quality
Solution Approach 2:
A rectified video portion is introduced as an intermediary element that bridges the gap between immersive and legacy formats. This rectified portion is generated by mapping the immersive video to a convex surface and then rectifying it to a flat rectangular format, serving as a compatibility layer
2Adaptability or versatility
If legacy video format is used for immersive devices, then legacy content can be displayed, but the video cannot be properly mapped on three-dimensional convex surfaces
Solution Approach 1:
The mapping from convex surface to rectilinear format is performed in advance during the video generation process. The rectified video portion is pre-computed and included in the enhancement layer, eliminating the need for real-time complex mapping operations on the rendering device
Solution Approach 2:
The solution transitions from three-dimensional convex surface mapping to two-dimensional rectilinear representation by introducing an intermediate rectified video layer. This dimensionality change allows legacy devices to display the content while immersive devices can reconstruct the 3D experience
3Device complexity
If a single video stream format is used for both legacy and immersive devices, then distribution is simplified, but one format cannot satisfy both rendering requirements
Solution Approach 1:
The video stream is designed with multi-functionality, containing both immersive video content and rectified video portions in a single unified format. This allows the same stream to be universally played by both legacy and immersive devices, with each device type utilizing the appropriate portion
Solution Approach 2:
The rectified video portion is nested within the enhancement layer of the immersive video stream. The enhancement layer contains references to specific portions of the base layer, creating a nested structure where the rectified content is embedded within the immersive content framework
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
The present disclosure relates to methods, apparatus or systems for formatting of backward compatible immersive video streams. At least one legacy rectangular video is captured from an immersive video obtained from a source (82). A set of camera control data are used to determine which parts of the immersive video will constitute legacy videos (84). These part are removed from the immersive video (83) and every prepared videos are packaged in a stream (85). The structure from the stream is a container. Information data about the location and size of removed parts may be added in the stream.