Derived Visual Tracks for Immersive Media Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional approaches to implement immersive media scenes require multiple processing pipelines on the client device, leading to heavy computational burdens and inefficient content delivery, especially for 3D content that demands compute-intensive processing steps.
Innovation Solution
The use of derived visual tracks in the ISO Base Media File Format (ISOBMFF) allows for performing composition and overlay derivation operations on input tracks, enabling scene tasks to be processed by decoders rather than the presentation engine, and shifting these operations to the server side for more efficient content delivery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple processing pipelines are implemented on the client device for immersive media scenes, then the content delivery can be maintained, but the computational burden on the client device increases significantly
Solution Approach 1:
The patent applies preliminary action by performing derivation operations on the server side before transmitting media data to the client device. The server pre-processes the media tracks and generates derived media data that is ready for playback, eliminating the need for complex client-side processing pipelines and significantly reducing the computational burden on client devices.
2Adaptability or versatility
If derivation operations are performed on the client side, then the processing flexibility is maintained, but the processing requirements and complexity on the client device increase
Solution Approach 1:
The patent extracts the complex derivation operations from the client device and relocates them to the server side. The server performs all necessary processing of media tracks to generate derived media data, while the client device only needs to handle the already-processed data, thereby reducing client device complexity while maintaining overall system flexibility through server-side adaptability.
3Manufacturing precision
If compute-intensive processing steps are performed on the client device for 3D content, then the content quality can be maintained, but the processing time and energy consumption increase
Solution Approach 1:
The patent applies preliminary action by pre-processing 3D content and generating derived media data on the server side before transmission. All compute-intensive processing steps are completed in advance, allowing the client device to receive ready-to-playback content without requiring significant processing time or energy, thus maintaining content quality while eliminating processing delays.
Data Source
AI summary
The techniques described herein relate to methods, apparatus, and computer readable media configured to generate media data for an immersive media experience. A set of parameters for processing a scene description for an immersive media experience are accessed. Multimedia data for the immersive media experience is accessed, including a plurality of media tracks, each media track comprising an associated series of samples of media data for a different component of the immersive media experience, and a derived track comprising a set of derivation operations to perform to generate a series of samples of media data for the client for the immersive media experience. A derivation operation is performed to generate a portion of media data for a derived media track, including processing the plurality of media tracks to generate a first series of samples of the media data for the immersive media experience, and outputting the derived media track.