Multi-Decoder Interface for Synchronized Immersive Media Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing APIs for handling immersive media data lack the ability to discover aggregate processing capability of multiple video decoders, fail to maintain minimum delay skew during decoding, and lack fine-granular control over decoder scheduling, leading to inefficiencies in processing multiple video streams.
Innovation Solution
Abstract extensions to decoder APIs are introduced to provide better support for immersive media, enabling logical grouping of samples and fine-granular control of decoder instances, mapped to OpenMAX IL and MediaCodec APIs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple decoders are used to support multiple formats, then format compatibility is improved, but device complexity increases
Solution Approach 1:
The system is divided into two independent parts: a format detection unit that identifies the media format, and a single decoder that processes the media based on the detected format. This segmentation allows the system to support multiple formats without requiring multiple decoders, thereby reducing device complexity while maintaining format compatibility.
Solution Approach 2:
A format detection unit is introduced as an intermediary component between the media input and the decoder. This intermediary detects the format of incoming media data and provides format information to the decoder, enabling a single decoder to handle multiple formats through format-specific processing modes rather than requiring multiple dedicated decoders.
2Measurement precision
If format detection is performed before decoding, then decoding accuracy is improved, but processing time increases
Solution Approach 1:
The format detection unit performs format detection as a preliminary action before the decoder processes the media data. By detecting the format in advance, the system ensures that the decoder is properly configured for the specific format, improving decoding accuracy. The detection is designed to be lightweight so that the time added is minimal.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An example device for decoding media data includes a memory configured to store the media data and one or more processors implemented in circuitry and communicatively coupled to the memory. The one or more processors are configured to determine whether two or more decoder instances are intended to be synchronized. The one or more processors are configured to, based on the two or more decoder instances being intended to be synchronized, control the two or more decoder instances so as to enable rendering of decoded data from each of the two or more decoder instances at a same presentation time.