Scene Audio Bitstream Metadata for Low-Latency Binaural Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing encoding methods for immersive audio in extended reality (XR) applications require significant time to obtain metadata for rendering binaural audio signals, leading to inefficiencies and reduced user experience.
Innovation Solution
An encoding method that writes scene audio signal metadata into a bitstream, allowing the decoder to parse and utilize this metadata directly, reducing the need for local generation and calculation, thereby speeding up the rendering process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If the decoder locally generates metadata of the scene audio signal, then the metadata can be obtained for rendering, but the time for obtaining binaural rendered signal through rendering is increased
Solution Approach 1:
The encoder pre-generates and embeds the metadata of the scene audio signal into the bitstream during the encoding phase. This preliminary action allows the decoder to directly extract the metadata without performing local generation, thereby reducing the time required for rendering and improving overall processing efficiency.
2Ease of operation
If the decoder obtains metadata from another electronic device, then the metadata can be acquired, but the complexity of the system increases and additional devices are required
Solution Approach 1:
The functionality of metadata generation and metadata provision are merged into a single component (the encoder). By embedding the metadata directly into the bitstream during encoding, the system eliminates the need for separate metadata generation devices or additional electronic devices, thereby reducing system complexity while maintaining ease of operation.
3Reliability
If the decoder locally generates metadata, then the system can operate independently, but the calculation amount for generating metadata increases
Solution Approach 1:
The computationally intensive metadata generation process is extracted from the decoder and performed instead by the encoder. The encoder extracts and embeds the pre-generated metadata into the bitstream, allowing the decoder to simply extract and use the metadata without performing heavy calculations, thereby reducing the calculation amount and energy consumption at the decoder while maintaining system independence.
Data Source
Figure 1A~1B
Figure 1C~1D
Figure 2A
AI summary
An encoding method and an electronic device are provided. The method includes: first obtaining a scene audio signal and metadata of the scene audio signal, where the scene audio signal describes a sound field of a sound source in a scene, the scene includes a plurality of microphones, the metadata includes first metadata, the first metadata is used in combination with a distance between two of the plurality of microphones to determine a virtual loudspeaker radius, the virtual loudspeaker radius and a virtual loudspeaker signal are used for rendering to obtain a rendered audio signal, and the virtual loudspeaker signal is generated based on a reconstructed audio signal of the scene audio signal; and encoding the scene audio signal and the metadata to obtain a bitstream. Therefore, a decoder side does not need to obtain the metadata of the scene audio signal from another electronic device, and does not need to locally generate the metadata of the scene audio signal, so that time for obtaining a binaural rendered signal through rendering can be reduced, thereby improving user experience.