Machine-Learning Audio Objectification for Real-Time Source Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional audio processing techniques struggle with converting non-object-based audio content into object-based audio content, which is laborious and expensive, and often result in decoding object-based audio to channel-based formats, limiting user interaction and manipulation.
Innovation Solution
Utilizing machine learning models, particularly deep neural networks, trained through supervised learning on short audio snippets to extract and manipulate individual audio sources in real-time or near real-time, enabling dynamic control of audio sources within non-object-based audio signals.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If non-object-based audio content is converted into object-based audio content using conventional techniques, then audio sources can be independently manipulated, but the conversion process is laborious and expensive
Solution Approach 1:
The patent replaces manual, mechanical audio conversion processes with an automated machine learning system. The neural network automatically separates mixed audio signals into individual source objects, eliminating the need for labor-intensive manual conversion while enabling independent manipulation of audio sources.
Solution Approach 2:
The audio conversion system performs self-service by automatically analyzing and separating audio sources without human intervention. The machine learning model autonomously processes channel-based audio and generates object-based output, making the conversion process independent and self-sufficient.
2Adaptability or versatility
If object-based audio is decoded to channel-based format for compatibility, then legacy audio content can be played, but user interaction and manipulation capabilities are limited
Solution Approach 1:
The system performs preliminary action by maintaining audio sources in object-based format throughout the processing pipeline. Instead of converting to channel-based format and losing object information, the system prepares and processes audio objects first, then selectively renders them to channel-based output only when needed for playback, preserving information throughout the process.
Solution Approach 2:
The patent segments the audio processing into distinct object-based components that can be independently manipulated. Each audio source is maintained as a separate object with its own properties, allowing users to interact with and modify individual sources before final mixing and playback, thus preserving information while enabling interaction.
3Productivity
If machine learning models are used to extract audio sources in real-time, then independent manipulation is enabled, but processing complexity increases
Solution Approach 1:
The patent implements a universal audio processing system that handles multiple functions: automatic source separation, object identification, parameter extraction, and real-time manipulation. This multi-functional approach consolidates complexity into a single integrated platform that can process various audio types and enable diverse manipulation capabilities simultaneously.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Techniques for dynamic audio objectification are described. Embodiments include providing a first audio snippet from an audio signal to a machine learning model trained based on audio snippets labeled with an audio source and receiving, from the machine learning model, a subset of the first audio snippet that is associated with the audio source. Embodiments include, after playing the reconstituted first audio snippet, receiving a changed configuration relating to the audio source. Embodiments include providing a second audio snippet from the audio signal to the machine learning model and receiving, from the machine learning model, a subset of the second audio snippet that is associated with the audio source. Embodiments include playing a reconstituted second audio snippet based on the subset of the second audio snippet and the changed configuration, wherein an audibly perceptible parameter of the audio source is changed in the reconstituted second audio snippet.