Immersive Audio Packaging for Codec-Agnostic 3D Stream Synchronization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing technologies lack a technique to express audio processing parameters for streaming 3D audio scenes and represent asynchronous multi-stream audio representations in a codec-agnostic manner, requiring multiple service and system processing for bitstream manipulation.

Innovation Solution

The implementation generates an audio package with codec-agnostic containers that decouple timing and audio parameters from the frame rate and codec, enabling asynchronous multi-stream audio representations and allowing for merging and processing without transcoding or retiming.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If conventional channel-based audio is used with one audio signal mapped to every speaker, then the audio can be played back on various devices, but it cannot represent immersive 3D audio scenes with asynchronous multi-stream audio representations

Engineering Contradiction:

Improveability to represent immersive 3D audio scenesVSAvoidcomplexity of audio processing and container structure

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The audio package is segmented into multiple independent audio streams, each representing different spatial components of the immersive audio scene. These streams are organized in a hierarchical container structure with codec-agnostic containers at the top level and codec-specific containers nested within, allowing independent processing and manipulation of each stream while maintaining overall scene coherence

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a new dimensional organization for audio data by adding temporal and spatial dimensions to the traditional channel-based approach. Audio streams are organized with time-stamp-based synchronization rather than frame-rate coupling, and spatial information is encoded as separate metadata layers, enabling asynchronous multi-stream representations without increasing processing complexity

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Ease of manufacture

If audio processing parameters are coupled with frame rate and codec, then the audio can be efficiently encoded, but bitstream manipulation such as merging and concatenation requires transcoding and retiming

Engineering Contradiction:

Improveease of bitstream manipulationVSAvoidtime required for transcoding and retiming operations

Core Design Contradiction:

Ease of manufactureVSLoss of time

Solution Approach 1:

The patent extracts audio processing parameters from their traditional coupling with frame rate and codec specifications. Processing parameters such as spatial position, volume, and panning are separated into independent metadata containers that can be manipulated without affecting the underlying audio codec or frame structure, enabling direct bitstream manipulation through copying and splicing operations

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The audio package is prepared in advance with codec-agnostic container structures that anticipate future manipulation needs. Time-stamp-based synchronization markers are embedded throughout the bitstream, and parameter containers are organized to facilitate easy merging and concatenation operations without requiring subsequent retiming or transcoding steps

Inventive Principle:

Principle #10Preliminary action

3Ease of operation

If automation data is tied to specific codec and timing structures, then the audio can be efficiently decoded, but the automation data cannot be independently manipulated or re-timed

Engineering Contradiction:

Improveindependent manipulation of automation dataVSAvoidcomplexity of container structure

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The patent introduces codec-agnostic container structures as intermediary layers between the audio codec and the automation data. These containers serve as neutral carriers that hold processing parameters independent of any specific codec implementation, allowing automation data to be manipulated, extracted, or modified without affecting the underlying audio encoding or requiring codec-specific processing

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The container structure is designed with universal functionality to accommodate multiple types of automation data and processing parameters within a single unified framework. The same container format can hold spatial position data for one stream and temporal synchronization data for another, eliminating the need for separate codec-specific automation structures and enabling cross-codec manipulation

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS12531075B2Immersive audio package

Publication Date: 2026.01.20 GOOGLE LLC

AI summary

A method including receiving first audio data, receiving second audio data, compressing the first audio data as first compressed audio data, compressing the second audio data as second compressed audio data, generating a codec dependent container including a parameter associated with compressing the first audio data, compressing the second audio data, a reference to the first compressed audio data, and a reference to the second compressed audio data, generating a codec agnostic container including at least one parameter representing time-varying data associated with playback of the first audio data and the second audio data, and generating an audio package including the codec dependent container and the codec agnostic container.