Headset-to-Head Rotation Transform for Spatial Audio

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing spatial audio platforms face challenges in maintaining accurate head pose tracking when the source device and headset are free to move relative to each other, impacting the user's perception of 3D spatial audio.

Innovation Solution

The method involves estimating a head to headset rotation transform by calculating gravity directions in both the source device and headset reference frames, using inertial measurement units (IMUs) and extended Kalman filters to maintain accurate tracking and rendering of spatial audio, even when the headset is positioned differently on the user's head.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If the source device and headset are free to move relative to each other, then the headset can be positioned at different locations on the user's head, but the accuracy of head pose tracking deteriorates

Engineering Contradiction:

Improveheadset position flexibilityVSAvoidhead pose tracking accuracy

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent introduces an intermediary computational model that estimates the rotation transform between headset and face reference frames using gravity directions from IMUs and relative motion data. This intermediary transformation model acts as a mediator to maintain accurate head pose tracking even when the headset moves relative to the source device, resolving the contradiction between headset position flexibility and tracking accuracy.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If the headset moves relative to the source device, then the user can wear the headset at different positions, but the stability of spatial audio perception deteriorates

Engineering Contradiction:

Improveheadset position flexibilityVSAvoidspatial audio stability

Core Design Contradiction:

Adaptability or versatilityVSStability of the object's composition

Solution Approach 1:

The patent implements a feedback mechanism where the system continuously estimates the rotation transform using IMU data from both the source device and headset, and uses this estimated transform to dynamically adjust the spatial audio rendering. This closed-loop feedback ensures that spatial audio remains stable and centered even when the headset moves relative to the source device, resolving the contradiction between headset position flexibility and spatial audio stability.

Inventive Principle:

Principle #23Feedback

3Measurement precision

If video camera tracking is used, then head pose can be tracked, but the system fails when source device and headset move relative to each other

Engineering Contradiction:

Improvehead pose tracking accuracyVSAvoidrelative motion tolerance

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent replaces the optical/mechanical video camera tracking system with an inertial sensing system using IMUs. Instead of relying on visual field-of-view constraints, the system uses gravity direction measurements and inertial motion data to compute the rotation transform between reference frames. This substitution enables the system to tolerate relative motion between source device and headset while maintaining head pose tracking accuracy.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This approach allows users to perceive stable and centered spatial audio regardless of the headset's position, ensuring that audio sources remain accurately localized in the 3D auditory space, even during head movements and changes in posture.

Implementation Method 1

estimating a first gravity direction in a source device reference frame for a source device; estimating a second gravity direction in a headset reference frame for a headset

Methodology Applied

Scientific EffectGravitation: Gravitation

Data Source

PatentUS11647352B2Head to headset rotation transform estimation for head pose tracking in spatial audio applications

Publication Date: 2023.05.09 APPLE INC

AI summary

In an embodiment, a method comprises: estimating a first gravity direction in a source device reference frame for a source device; estimating a second gravity direction in a headset reference frame for a headset; estimating a rotation transform from the headset frame into a face reference frame using the first and second estimated gravity directions, a rotation transform from a camera reference frame to the source device reference frame, and a rotation transform from the face reference frame to the camera reference frame; estimating a relative position and attitude using source device motion data, headset motion data and the rotation transform from the headset frame to the face reference frame; using the relative position and attitude to estimate a head pose; and using the estimated head pose to render spatial audio for playback on the headset.