Real-Time Speech Analysis for Selective Audio Volume Adjustment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio processing systems cannot adjust specific sounds, such as musical instrument sounds or human voices, in real time, limiting user audio enjoyment.
Innovation Solution
A real-time audio processing system utilizing a speech analysis model that processes audio in real time by extracting target sounds and adjusting their volumes, achieved through a method involving Fourier transforms, mask information, and a loss function to train the model for selective audio extraction and adjustment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If audio processing is performed in real-time, then audio enjoyment is enhanced, but processing speed and latency requirements increase system complexity
Solution Approach 1:
The audio signal is divided into multiple frames, with each frame processed independently through the speech analysis model. This segmentation allows parallel processing of different audio segments, improving overall processing speed while maintaining manageable system complexity through modular architecture
Solution Approach 2:
The speech analysis model performs preliminary analysis of audio frames to extract speech components and characteristics before final audio output is generated. This preliminary processing enables real-time decision-making about which audio components to enhance or suppress, achieving real-time audio enhancement without excessive system complexity
2Adaptability or versatility
If speech analysis model is used to extract target sounds, then selective audio adjustment is enabled, but computational resources and processing time increase
Solution Approach 1:
The speech analysis model applies different processing qualities to different components of the audio signal. Speech components are extracted and processed with high fidelity using the full model capability, while non-speech components are handled with simpler processing, optimizing computational resource allocation based on the local characteristics of each audio segment
Solution Approach 2:
The system dynamically adjusts processing parameters such as model confidence thresholds and processing intensity based on the detected speech characteristics. When speech is clearly detected, the full analytical capability is applied; when speech is ambiguous or absent, processing is simplified or skipped, reducing overall computational resource consumption while maintaining selective adjustment capability
Data Source
AI summary
An audio real-time processing system, an audio real-time processing program product and method for training speech analysis model are provided. The speech analysis model is firstly trained to obtain, from an original audio, mask information which is used to mask the original audio to get a target audio. The system obtains a plurality of analyzed audio according to the target audio and the original audio, obtains repeated audio section according to the plurality of the analyzed and output the repeated audio section.