Video Audio Channel Gain Control Using On-Screen Source Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile devices lack studio-quality audio hardware, making it difficult to capture high-quality audio despite advancements in video capture technology.
Innovation Solution
A computer-implemented method using machine-learning models to separate audio into channels, classify audio sources, and adjust volume levels based on their presence and type in the video, enhancing desired audio sources and reducing unwanted noise.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If professional audio hardware (directional microphones, tuned microphones) is used, then audio quality is improved, but device size and cost increase
Solution Approach 1:
The patent replaces physical audio hardware (directional microphones, acoustic lenses) with software-based audio separation and enhancement algorithms. The system uses machine learning models to separate audio sources and enhance desired sounds without requiring specialized microphone arrays or acoustic structures, thus maintaining audio quality while avoiding the size and cost constraints of professional audio equipment.
2Manufacturing precision
If professional audio hardware is used, then audio quality is improved, but device cost increases
Solution Approach 1:
The invention substitutes expensive professional audio hardware with software-based processing. The system uses machine learning models for audio separation, source identification, and enhancement that can be implemented on standard mobile device processors, eliminating the need for costly directional microphones, acoustic lenses, and other professional audio components.
Solution Approach 2:
The patent employs software algorithms that can be updated and improved without hardware changes. The audio enhancement functionality is implemented through software models that can be deployed on existing mobile devices, avoiding the need for expensive, specialized hardware components that increase manufacturing costs.
3Manufacturing precision
If audio enhancement algorithms are applied, then audio quality is improved, but computational complexity increases
Solution Approach 1:
The patent segments the audio enhancement task into distinct processing stages: audio separation into individual sources, classification of audio sources, identification of on-screen audio sources, and selective enhancement. This segmentation allows each sub-task to be handled by specialized machine learning models, improving efficiency and reducing overall computational complexity compared to a monolithic approach.
Solution Approach 2:
The system performs preliminary audio separation and source classification before applying enhancement algorithms. By pre-identifying and categorizing audio sources, the system can apply enhancement only to relevant channels, reducing computational load compared to processing all audio channels uniformly.
Data Source
AI summary
A media application obtains a video that includes an audio portion. The media application separates the audio portion into a plurality of channels, where each channel corresponds to a particular audio source. An on-screen classifier model obtains an indication of whether the particular audio source for each channel is depicted in the video. An audio-type classifier model determines, an auditory object classification for each channel. The media application determines a respective gain for each channel based on the indication of whether the particular audio source for the channel is depicted in the video and the auditory object classification for the channel. The media application modifies each channel by applying the respective gain. The media application mixes the modified channels with the audio portion to generate a combined audio.