Adaptive Streaming Metadata for Nonlinear Media Playback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current adaptive streaming technologies for nonlinear media, such as spherical video, lack flexibility in authoring and personalization, are not scalable for large-scale content delivery, and are incompatible with state-of-the-art streaming protocols like HTTP adaptive streaming, limiting user interaction and media experience customization.
Innovation Solution
A method for adaptive streaming that uses a server to select and transmit metadata based on navigation information and spatial playback data, allowing clients to switch between different media paths and enabling flexible authoring and dynamic content insertion, compatible with HTTP adaptive streaming protocols like MPEG DASH.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a content creator system links 360 video clips using hotspots and creates an interactive 360 video file by transcoding the linked clips into a single interactive 360 video file, then the 360 video content can be played back via different playback paths, but the technology does not generate content that can be distributed in a scalable manner and is compatible with state-of-the-art streaming protocols like HTTP adaptive streaming protocols (MPEG DASH and HLS) and content distribution systems like content distribution networks (CDNs)
Solution Approach 1:
The system segments the interactive 360 video content into multiple video fragments (chunks or media segments) that can be independently distributed and cached in CDNs. Each fragment corresponds to a specific时间段 and can be selectively streamed based on user navigation, enabling both playback path flexibility and scalable distribution through standard HTTP adaptive streaming protocols.
Solution Approach 2:
The system introduces a manifest file as an intermediary that describes the structure of the interactive video content, including video fragments, their temporal relationships, and spatial playback regions. This manifest file enables HTTP adaptive streaming clients to retrieve and playback video fragments in the correct sequence without requiring custom protocol implementations, thus achieving compatibility with standard CDNs and streaming protocols.
2Stability of the object's composition
If a content creator system creates an interactive 360 video file with fixed links and content, then the video can be played back through defined paths, but the technology does not allow flexible authoring and dynamic introduction of new linked pieces of 360 video or removal of links to existing 360 video during playback
Solution Approach 1:
The system implements dynamic playback paths by allowing the client to request different video fragments based on user navigation actions (e.g., hotspot selections). The manifest file structure supports multiple possible playback paths, and the server can dynamically serve different fragments depending on the current playback state and user interactions, enabling flexible authoring while maintaining video structure integrity.
Solution Approach 2:
The system prepares multiple video fragments and their corresponding metadata in advance during the authoring phase, organizing them in a manifest file that describes all possible playback paths. This preliminary organization allows the system to support dynamic content modification during playback without requiring real-time transcoding or re-authoring, as all necessary video segments are pre-segmented and ready for selective delivery.
3Productivity
If adaptive streaming divides source video into sequentially ordered video fragments with monotonically increasing sequence numbers, then the video can be streamed adaptively, but the technology has limited flexibility in authoring, amending, and personalizing playback paths and does not allow repetitive playback of video fragments that have already been played back earlier in the playback path
Solution Approach 1:
The system makes video fragments universally playable by designing them as independent, self-contained units that can be played in multiple contexts. Each video fragment is marked with metadata indicating its temporal position and relationships to other fragments, allowing the same fragment to be played both in sequential order and in repetitive or non-linear patterns based on user navigation, thus achieving both streaming efficiency and playback path customization.
4Ease of operation
If the streaming process uses a manifest file to describe video fragments and their sequence numbers, then HTTP adaptive streaming can retrieve and playback video fragments in accordance with sequence numbers, but the technology does not provide a scheme for flexible insertion of different alternative segments in an already existing playback scheme
Solution Approach 1:
The system adds a spatial dimension to the traditional sequential video playback by introducing spatial playback region descriptors that link regions in video frames to alternative video fragments. This allows the manifest file to describe not only temporal sequencing but also spatial relationships, enabling flexible insertion of alternative segments based on user navigation within the video scene while maintaining compatibility with HTTP adaptive streaming protocols.
Data Source
Figure 1~2B
Figure 3~4
Figure 5
AI summary
A method for adaptive streaming of video is described, wherein the method comprises: transmitting first metadata to a client apparatus, the first metadata defining a first adaptation set, an adaptation set defining one or more sets of segment identifiers, each set of segment identifiers defining a plurality of linearly playable media segments of a predetermined resolution, each media segment in a set of media segments having an identical playback duration, the first metadata enabling a client apparatus to request media segments and sequentially playout the media segments; receiving navigation information from the client apparatus, the navigation information defining at least a first region, in the video frames of the first adaptation set; selecting second metadata defining a second adaptation set on the basis of spatial playback information and a spatial distance or a spatial overlap between the first region and a second region in video frames of the first adaptation set, wherein the spatial playback information signals the server apparatus that the second region is linked to media segments of the second adaptation set; and, transmitting the second metadata or information for retrieving the second metadata to the client apparatus, the second metadata enabling the client apparatus to playback media segments of the second adaptation set after the playback of media segments of the first adaptation set.