Directional Audio Stabilization for Immersive Services
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current audio codecs, such as the 3GPP EVS codec, lack the necessary features for immersive audio experiences, including support for multi-channel functionality, low latency, and the ability to handle undesired movements of the microphone system capturing directional audio, which can cause spatial rotation and translation of the rendered scene.
Innovation Solution
A device and method for capturing and encoding directional audio that compensates for undesired movements by modifying the directional properties of the audio based on spatial data from the microphone system, and a corresponding rendering device that decodes and renders the modified audio, ensuring a stable audio experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If spatial data is transmitted to compensate for microphone movements, then audio scene stability is improved, but transmission bandwidth and device complexity increase
Solution Approach 1:
The patent applies preliminary action by modifying the directional audio properties at the capturing device before transmission. The encoder device adjusts the audio scene compensation in advance based on detected microphone movements, so that the rendered audio scene remains stable without requiring complex real-time processing at the rendering device. This resolves the contradiction by performing the stabilization action upfront, reducing the computational burden during playback.
2Adaptability or versatility
If directional audio is captured with moving microphone systems, then immersive audio coverage is improved, but spatial rotation and translation of the rendered scene occur causing motion sickness
Solution Approach 1:
The patent applies preliminary anti-action by detecting microphone system movements and pre-compensating the directional audio properties before transmission. The encoder device modifies the audio data to counteract the effects of microphone movement, so that when the audio is rendered, the spatial rotation and translation are compensated and do not cause motion sickness. This allows the microphone to move freely for immersive coverage while preventing the harmful effects in the final audio output.
3Measurement precision
If spatial data is transmitted for audio scene compensation, then audio rendering accuracy is improved, but transmission bandwidth increases
Solution Approach 1:
The patent applies the taking out principle by extracting only the essential movement compensation information from the full spatial data. Instead of transmitting complete spatial coordinates and orientation data, the encoder device processes the directional audio to embed only the necessary compensation parameters. This selective extraction maintains audio rendering accuracy while significantly reducing the bandwidth required for transmission.
Data Source
AI summary
The disclosure herein generally relates to capturing, acoustic pre-processing, encoding, decoding, and rendering of directional audio of an audio scene. In particular, it relates to a device adapted to modify a directional property of a captured directional audio in response to spatial data of a microphone system capturing the directional audio. The disclosure further relates to a rendering device configured to modify a directional property of a received directional audio in response to received spatial data.