Spatial Audio Signal Alignment for Low-Latency Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio processing systems face challenges in accounting for delays between audio signals captured by microphones located at different positions, which can lead to latency and perceptible artifacts in spatial audio rendering, especially in mediated reality applications.
Innovation Solution
A method and apparatus that temporally align audio signals captured by local and distant microphones within a threshold time, discarding signals outside this threshold to minimize latency and prevent artifacts, using a processing device to adjust delays and perform spatial audio processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple audio signals are processed by separate processing circuits, then processing capability is improved, but device complexity increases
Solution Approach 1:
The patent combines multiple audio processing circuits into a single integrated processing circuit that can handle multiple audio signals simultaneously. This merging approach maintains the processing capability of multiple signals while reducing the overall device complexity by eliminating redundant circuit components and interconnections.
Solution Approach 2:
The processing circuit is designed with multi-functionality to handle different types of audio signals (microphone signals, auxiliary input signals, etc.) and perform various processing operations (noise suppression, echo cancellation, volume control) within a single unified architecture, thereby improving productivity without proportionally increasing device complexity.
2Reliability
If noise suppression processing is applied to auxiliary input signals, then sound quality is improved, but processing time increases
Solution Approach 1:
The system performs preliminary classification of auxiliary input signals to identify their types (music, speech, etc.) and characteristics before applying noise suppression processing. This preliminary action allows the system to pre-select appropriate processing parameters and algorithms, thereby improving sound quality while minimizing processing time by avoiding unnecessary or overly complex processing steps.
Solution Approach 2:
The noise suppression processing is applied selectively based on signal characteristics rather than uniformly to all auxiliary input signals. The system applies processing only when and where needed, using partial action to maintain sound quality improvement while reducing overall processing time by avoiding redundant processing on signals that don't require it.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method, apparatus and computer program, the method comprising: obtaining a first audio signal emitted by an audio source, wherein the first audio signal is captured by a first microphone located at a first position; obtaining at least one second audio signal emitted by the same audio source, wherein the at least one second audio signal is captured by one or more second microphones located at one or more second positions which are different to the first position; determining if one or more of the second audio signals were obtained within a threshold time; and if one or more second audio signals were obtained within the threshold time causing the one or more second audio signals that were obtained within the threshold time to be processed for rendering spatial audio to a user; and if one or more second audio signals were not obtained within the threshold time causing, at least part of, the one or more second audio signals that were not obtained within the threshold time to be discarded.