Live Audio Track Addition to Digital Video Streams

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing video streaming technologies do not efficiently allow for the addition of multiple audio tracks in real-time to a live video stream, limiting the ability to provide localized commentary or audio customization for global broadcasts without incurring significant production and travel costs.

Innovation Solution

A system that decodes live video streams, adds new audio tracks through mixing with existing audio, synchronizes them using timestamps, and encodes the mixed audio stream for real-time publication, enabling consumers to download the video with new audio tracks on a slight delay.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If traditional video streaming technology is used, then the system is simple and production costs are low, but the ability to add multiple audio tracks in real-time is limited

Engineering Contradiction:

Improveability to add multiple audio tracksVSAvoidsystem complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system separates audio processing from video processing by decoding only the audio portion of the video stream, mixing it with additional audio tracks, and then re-encoding. This segmentation allows independent audio manipulation without affecting the video stream, enabling multiple audio track addition while maintaining system manageability.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces an intermediary audio mixing system that sits between the video source and the output stream. This intermediary component decodes the audio, mixes it with additional commentary or language tracks, and feeds the mixed audio back into the stream, providing the versatility to add multiple audio tracks without completely redesigning the entire streaming system.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Ease of manufacture

If commentators are located at the event site for live commentary, then the audio quality and synchronization are good, but the production and travel costs are high

Engineering Contradiction:

Improveproduction costVSAvoidaudio synchronization

Core Design Contradiction:

Ease of manufactureVSReliability

Solution Approach 1:

The system performs preliminary decoding of the audio stream at the remote location where commentators work from their home countries. By pre-decoding the audio and preparing it for mixing before the final output, the system enables remote commentators to contribute without being physically present at the event, reducing travel costs while maintaining synchronization through timestamp-based mixing.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system uses feedback mechanisms where the mixed audio stream is continuously monitored and adjusted. Timestamps are applied to ensure proper synchronization between the original audio and the additional commentary tracks, allowing remote commentators to maintain accurate timing without being physically present at the event site.

Inventive Principle:

Principle #23Feedback

3Adaptability or versatility

If audio tracks are added in real-time to live video streams, then the audio customization capability is improved, but the processing time and computational resources increase

Engineering Contradiction:

Improveaudio customization capabilityVSAvoidcomputational resources

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The system extracts only the audio portion from the video stream for processing, leaving the video stream unchanged. By taking out just the audio component for decoding, mixing, and re-encoding, the system reduces the computational burden compared to processing the entire video stream, while still achieving the goal of adding multiple audio tracks with full customization capability.

Inventive Principle:

Principle #2Taking out (Extraction)

4Productivity

If multiple audio tracks are mixed and encoded in real-time, then the audio track availability is improved, but the encoding complexity and processing load increase

Engineering Contradiction:

Improveaudio track availabilityVSAvoidencoding complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system performs partial encoding by only re-encoding the mixed audio portion rather than the entire video stream. This partial action approach provides rapid audio track availability for consumers while keeping encoding complexity manageable by focusing computational resources only on the audio component that requires mixing and customization.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS8643779B2Live audio track additions to digital streams

Publication Date: 2014.02.04 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

Audio tracks may be added to a live video stream using one or more mixers that may decode the audio from a video stream, mix an additional audio stream, and encode the mixed audio stream. The system may make the new audio stream available for live download by consumers. The mixers may include a video playback mechanism to view the live video stream, as well as an audio capture and mixing system that may create a new audio track that may be synchronized with the video stream using timestamps. A consumer device may download and consume the video with the new audio track on a slight delay from the live video stream.