Binaural Downmix Separation for Low-Latency Head Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing binaural processing techniques for wireless audio systems face challenges in maintaining low latency and efficient head-tracking due to channel limitations and high latency in wireless transmission protocols, which degrade the auditory experience.
Innovation Solution
Separate channels into head-tracked and fixed channels, performing binaural downmixing on fixed channels at the source device and head-tracking on head-tracked channels at the output device, using encoding techniques to transmit within channel limitations, such as mid-side encoding and Stereo Quadraphony, to maintain low latency and computational efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If binaural processing is performed on the source device prior to transmitting audio data to headphones via a wireless protocol, then channel limitations of wireless protocols are satisfied, but head-tracking latency increases beyond perceptual thresholds
Solution Approach 1:
The patent segments the audio processing into two distinct parts: binaural downmixing is performed on a first subset of channels at the source device, while head-tracking processing is performed on a second subset of channels at the output device. This segmentation allows each processing stage to be optimized independently - the source device handles channel-compliant processing while the output device handles low-latency head-tracking using locally captured sensor data, thereby resolving the contradiction between wireless protocol channel limitations and head-tracking latency requirements
Solution Approach 2:
The patent introduces an intermediary approach where the source device performs preliminary binaural downmixing on certain channels before transmission, creating a baseline audio signal that complies with wireless channel limitations. The output device then acts as an intermediary that receives this pre-processed signal and adds head-tracking processing locally, avoiding the need to transmit high-latency head-tracking data wirelessly while still achieving low-latency spatial audio effects
2Adaptability or versatility
If head-tracking data is sent over a wireless protocol, then binaural processing with head-tracking can be achieved, but round-trip latency exceeds perceptual thresholds
Solution Approach 1:
The patent enables the output device to serve itself by performing head-tracking processing locally using sensors (such as IMUs) that are already present in the headphones. Instead of relying on the source device to process head-tracking data, the output device autonomously captures its own positional data and applies binaural processing with head-tracking locally, eliminating the need to transmit head-tracking data wirelessly and thereby resolving the latency issue while maintaining full head-tracking capability
3Loss of time
If all channels are processed for head-tracking at the output device, then low latency is achieved, but computational load increases
Solution Approach 1:
The patent segments the channel processing workload by designating a first subset of channels for binaural downmixing at the source device and a second subset of channels for head-tracking processing at the output device. This segmentation reduces the computational load at the output device, as it only needs to perform head-tracking on the second subset rather than all channels, while still achieving low latency for the head-tracked channels and distributing the overall processing burden efficiently across both devices
Data Source
AI summary
Techniques for separating binaural downmixing and head tracking processing between a source device and an output device are described. Embodiments include receiving, by a source device, an audio signal comprising a plurality of channels and selecting a first subset of the plurality of channels as head-tracked channels. Embodiments include performing, by the source device, binaural downmixing on a second subset of the plurality of channels that is different than the first subset of the plurality of channels to produce a downmixed second subset of the plurality of channels. Embodiments include transmitting, by the source device, the first subset of the plurality of channels and the downmixed second subset of the plurality of channels to an output device. Embodiments include performing, by the output device, binaural downmixing on the first subset of the plurality of channels based on positional data captured via one or more sensors associated with the output device.