Immersive Audio Loudness Control for Multi-Speech Scenes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio processing technologies in virtual and augmented reality applications fail to effectively adjust sound levels to create an immersive experience, particularly when multiple speech signals are present in a scene, leading to unrealistic audio perception and interaction.
Innovation Solution
A method and apparatus for audio processing that determines a loudness adjustment for an adjusted speech signal based on multiple speech signals in a scene, encoding this information in a bitstream, and adjusting sound levels accordingly to match a reference signal, using techniques such as binaural rendering and loudness controllers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional audio processing is used in virtual reality applications, then the system complexity remains low, but the audio realism and immersive experience deteriorate
Solution Approach 1:
The patent applies preliminary action by pre-calculating and encoding loudness adjustment parameters in the bitstream during the content creation phase. The loudness metadata is prepared in advance based on scene analysis, allowing the rendering device to directly apply pre-determined adjustments without performing complex real-time calculations, thus achieving realistic audio perception while keeping the runtime system complexity low.
2Measurement precision
If loudness adjustment parameters are encoded for each individual speech signal, then the audio perception accuracy improves, but the bitstream data volume and processing complexity increase
Solution Approach 1:
The patent merges multiple individual loudness adjustment parameters into a single scene-level loudness adjustment parameter. Instead of encoding separate adjustments for each speech signal, the system calculates one comprehensive loudness parameter for the entire scene based on the aggregate loudness of all speech signals. This significantly reduces the bitstream data volume while maintaining sufficient audio perception accuracy through the unified adjustment.
3Ease of operation
If scene-level loudness adjustment is implemented, then the processing complexity is reduced, but the precision of individual speech signal loudness control decreases
Solution Approach 1:
The patent applies local quality by differentiating the control approach between scene-level and individual signal levels. The scene-level adjustment provides coarse control for overall balance, while the individual speech signal loudness information is preserved in the bitstream to enable fine-grained local adjustments when needed. This hierarchical approach maintains processing simplicity at the scene level while preserving the capability for precise individual control when required.
Data Source
AI summary
Aspects of the disclosure provide methods and apparatuses for audio processing. In some examples, an apparatus of audio coding includes processing circuitry. The processing circuitry decodes, from a coded bitstream, information indicative of an adjusted speech signal and a loudness adjustment to the adjusted speech signal. The adjusted speech signal is indicated in an association with multiple speech signals in a scene of an immersive media application. The processing circuitry determines a plurality of loudness adjustments to sound signals including the multiple speech signals in the scene based the plurality of loudness adjustment to the adjusted speech signal, and generates the sound signals in the scene based on the loudness adjustments to the sound signals.