Scene Audio Decoding With Embedded Metadata for Faster Binaural Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for immersive audio in extended reality (XR) applications require significant time to obtain metadata for rendering binaural audio signals, as they often rely on obtaining and locally generating scene audio signal metadata, which hinders efficient decoding and user experience.
Innovation Solution
A decoding method that includes encoding scene audio signals with metadata, allowing the decoder to parse and utilize pre-encoded metadata directly, reducing the need for local generation and improving efficiency by determining virtual loudspeaker parameters for rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If the decoder obtains metadata from another electronic device or locally generates it, then the rendering process can be completed, but the time required to obtain the rendered audio signal increases
Solution Approach 1:
The metadata of the scene audio signal is pre-encoded into the bitstream by the encoder. This preliminary action allows the decoder to directly parse and obtain the metadata from the bitstream without needing to acquire it from another electronic device or generate it locally, thereby reducing the time required to obtain the rendered audio signal and improving rendering efficiency
2Productivity
If the decoder locally generates the metadata of the scene audio signal, then the rendering process can be completed, but the calculation amount increases
Solution Approach 1:
The metadata that would otherwise need to be calculated by the decoder is extracted and embedded into the bitstream by the encoder. This extraction transfers the computational burden from the decoder to the encoder, allowing the decoder to simply parse the pre-computed metadata from the bitstream, thereby reducing the calculation amount and improving rendering efficiency
Data Source
Figure 1A~1B
Figure 1C~1D
Figure 2A
AI summary
A decoding method and an electronic device are provided. The method includes: first receiving a bitstream; then parsing the bitstream to obtain metadata of a scene audio signal; and obtaining a reconstructed audio signal of the scene audio signal based on the bitstream, where the scene audio signal describes a sound field of a sound source in a scene, the scene includes a plurality of microphones, the metadata includes first metadata, the first metadata is used in combination with a distance between two of the plurality of microphones to determine a virtual loudspeaker radius, the virtual loudspeaker radius and a virtual loudspeaker signal are used for rendering to obtain a rendered audio signal, and the virtual loudspeaker signal is generated based on the reconstructed audio signal. In this way, a decoder side does not need to obtain the metadata of the scene audio signal from another electronic device, and does not need to locally generate the metadata of the scene audio signal, so that a calculation amount of generating the metadata of the scene audio signal is reduced. Therefore, time for obtaining a binaural rendered signal through rendering can be reduced, thereby improving user experience.