Adaptive Immersive Media Distribution for Heterogeneous Clients

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The lack of a coherent end-to-end ecosystem for distributing immersive media over commercial networks is hindered by the absence of a single standard representation, heterogeneity of immersive media devices, and the complexity of adapting media for real-time and non-real-time distribution, particularly for devices capable of displaying multiple unique views.

Innovation Solution

A network-based media distribution system that adapts immersive media sources to match the specific characteristics of client end-point devices, including interpolating additional views using neural networks, to create a suitable format for both legacy and immersive media displays, supporting both timed and untimed media.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If immersive media is distributed to heterogeneous client devices without adaptation, then distribution complexity is reduced, but device compatibility and user experience quality deteriorate

Engineering Contradiction:

Improvedistribution complexityVSAvoiddevice compatibility

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

The system changes media parameters (resolution, frame rate, bit rate, view configuration) based on client device characteristics. The packaging process selects and adapts media representations with different parameters to match client capabilities, resolving the contradiction between distribution simplicity and device compatibility.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The immersive media is segmented into multiple representations with different characteristics (e.g., different resolutions, view configurations). The packaging process divides the media into selectable segments that can be independently adapted to different client devices, maintaining compatibility across heterogeneous devices while simplifying the overall distribution architecture.

Inventive Principle:

Principle #1Segmentation

2Manufacturing precision

If immersive media is adapted for each client device, then user experience quality is improved, but processing time and system complexity increase

Engineering Contradiction:

Improvemedia adaptation qualityVSAvoidprocessing time

Core Design Contradiction:

Manufacturing precisionVSLoss of time

Solution Approach 1:

The packaging process performs preliminary adaptation actions by pre-packaging media with multiple representations and metadata during the ingestion phase. This preliminary action prepares the media in advance, reducing real-time processing requirements and minimizing loss of time during actual distribution while maintaining high adaptation quality.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The packaging process acts as an intermediary between the source immersive media and client devices. It performs the adaptation work in this intermediate stage, creating packaged media that bridges the gap between source and destination, thereby reducing the processing burden on both ends and minimizing overall processing time.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If multiple media representations are packaged for different devices, then device compatibility is improved, but data volume and storage requirements increase

Engineering Contradiction:

Improvedevice compatibilityVSAvoiddata volume

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The packaging process creates a universal packaged media structure that can serve multiple device types. By organizing media representations with standardized metadata and selection mechanisms, a single packaged file can adapt to various clients, reducing the need for separate media files for each device type and thereby reducing overall data volume.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

Multiple media representations are merged into a single packaged media structure with associated metadata. Instead of distributing separate files for different devices, the system combines multiple representations into one unified package, reducing storage requirements and data transmission volume while maintaining compatibility across devices.

Inventive Principle:

Principle #5Merging (Combining)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

Enables simultaneous support for a variety of legacy and immersive media-capable devices, facilitating flexible distribution formats for real-time and on-demand use cases, and accommodating heterogeneous client end-points with varying capabilities and requirements.

Implementation Method 1

interpolating additional views using neural networks

Methodology Applied

Scientific EffectNeural network interpolation:

Data Source

PatentUS11570227B2Set up and distribution of immersive media to heterogenous client end-points

Publication Date: 2023.01.31 TENCENT AMERICA LLC

AI summary

A method, computer program, and computer system is provided for streaming immersive media. Information corresponding to a characteristic of a client associated with media content is received based on the transfer of one or more tokens. A packaging process is associated with the media content using the one or more tokens. The media content is adapted according to the characteristic of the client based on the associated packaging process.