Virtual Loudspeaker Objects for Acoustic Scene Playback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for reproducing acoustic scenes require explicit knowledge of room properties and source locations, making them impractical for high-quality walkthroughs in real acoustic scenes, especially since object-based solutions necessitate numerous microphones or source separation techniques, which are difficult to implement effectively.
Innovation Solution
The method involves assigning Virtual Loudspeaker Objects (VLOs) to microphone setups within a virtual free field, allowing for continuous variation of the virtual listening position without needing explicit information about sound sources or room properties, using encoded data streams that encode spatial distribution and reverberations, enabling interactive traversal of recorded acoustic scenes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If object-based audio rendering is used to render a walkthrough, then the playback perspective can be adapted by changing the arrangement of recorded sources, but explicit knowledge of room properties, source locations and source properties is required
Solution Approach 1:
The patent creates a virtual copy of the acoustic scene by assigning Virtual Loudspeaker Objects (VLOs) to microphone setups. Instead of requiring explicit knowledge of physical sources and room properties, the system copies the acoustic scene's characteristics into a virtual representation that can be freely explored. The VLOs encode the spatial distribution and reverberation properties, allowing perspective adaptation without needing to know the original scene's detailed properties.
Solution Approach 2:
The patent introduces VLOs as an intermediary between the recorded acoustic scene and the playback system. These VLOs serve as abstract sound output objects that mediate the transformation of recorded microphone signals into spatial audio experiences. By using VLOs as intermediaries, the system avoids the need for direct analysis and reproduction of physical sources and room properties.
2Measurement precision
If object-based solutions are implemented, then source positions can be obtained after projecting directions onto a convex hull, but this causes signal degradation and only works for small position changes
Solution Approach 1:
The patent implements dynamic perspective traversal by allowing continuous movement of the virtual listening position through the acoustic scene. The VLO parameters are dynamically adjusted based on the desired virtual listening position, enabling large position changes without signal degradation. This dynamic approach replaces the static convex hull projection method with a flexible system that maintains signal quality across the entire scene.
3Adaptability or versatility
If source separation techniques are used to extract individual sources from a mix, then object-based representation can be obtained, but this is a difficult task requiring many microphones close to all desired sources
Solution Approach 1:
The patent uses a limited number of microphone setups recorded at specific locations within the scene, rather than requiring many microphones close to all sources. By strategically placing microphones and using VLOs to represent the acoustic scene, the system achieves object-based representation with fewer recording resources. The VLOs encode the spatial information and reverberation, providing partial but sufficient action for creating immersive audio experiences.
Data Source
Figure 1
Figure 2a
Figure 2b
AI summary
A method for acoustic scene playback comprises: providing recording data comprising microphone signals of one or more microphone setups positioned within an acoustic scene and microphone metadata of the one or more microphone setups, wherein each of the one or more microphone setups comprises one or more microphones and has a recording spot which is a center position of the respective microphone setup; specifying a virtual listening position, wherein the virtual listening position is a position within the acoustic scene; assigning each microphone setup of the one or more microphone setups one or more Virtual Loudspeaker Objects, VLOs, wherein each VLO is an abstract sound output object within a virtual free field; generating an encoded data stream based on the recording data, the virtual listening position and VLO parameters of the VLOs assigned to the one or more microphone setups; and decoding the encoded data stream based on a playback setup, thereby generating a decoded data stream; and feeding the decoded data stream to a rendering device, thereby driving the rendering device to reproduce sound of the acoustic scene at the virtual listening position. A playback apparatus and a computer program for performing the method for acoustic scene playback are also described.