Predictive Azimuth Correction for VR Audio Processing Delays

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In virtual reality and augmented reality applications using head-mounted displays, processing delays cause deviations in the direction of sound reproduction, leading to distortion of the intended sound space, as sound waves propagate with significant delays affecting head motion synchronization.

Innovation Solution

A signal processing device and method that predict the relative azimuth of virtual sound sources based on propagation and processing delays, using head angular velocity and acceleration information to correct the head-related transfer functions and generate binaural-room impulse responses (BRIR) for accurate sound reproduction during head motion.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If binaural reproduction is implemented in head-mounted display, then immersion is enhanced, but processing delay causes deviation in sound reproduction direction

Engineering Contradiction:

ImproveimmersionVSAvoidsound reproduction direction accuracy

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The system performs preliminary action by predicting the future azimuth angle based on current head orientation and predicted head motion before the sound is actually reproduced. This prediction compensates for the processing delay, ensuring that the sound is reproduced in the correct direction when it reaches the listener's ear.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system uses feedback from head motion detection to continuously update the azimuth angle prediction. By monitoring actual head motion and comparing it with predicted motion, the system adjusts the sound reproduction parameters to maintain accurate spatial positioning despite processing delays.

Inventive Principle:

Principle #23Feedback

2Manufacturing precision

If sound wave propagation delay is accounted for, then sound space distortion is prevented, but processing complexity increases

Engineering Contradiction:

Improvesound space accuracyVSAvoidprocessing complexity

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The system changes the parameter of azimuth angle dynamically based on the distance to the virtual sound source and predicted head motion. By adjusting the azimuth angle parameter in advance according to propagation delay characteristics, the system maintains sound space accuracy without requiring complex real-time processing during sound reproduction.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS12108242B2Signal processing device and signal processing method

Publication Date: 2024.10.01 SONY GROUP CORP

AI summary

There is provided a signal processing device that includes a relative azimuth prediction unit that predicts, on the basis of a delay time in accordance with a distance from a virtual sound source to a listener, a relative azimuth of the virtual sound source when a sound of the virtual sound source reaches the listener, and a binaural-room impulse response (BRIR) generation unit that acquires a head-related transfer function of the relative azimuth for each one of a plurality of the virtual sound sources and generates a BRIR on the basis of a plurality of the acquired head-related transfer functions.