Ear Tracking via Facial Landmark Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing camera-based head and ear tracking systems for personal and near-field audio systems face challenges in accurately scaling pixel data to three-dimensional ear positions due to variations in inter-ocular distance and other presumptive dimensions, leading to unacceptable noise and crosstalk cancellation in audio systems.
Innovation Solution
A computer-implemented method for audio processing that involves acquiring images of a user, processing these images to identify face landmarks, selecting landmark pairs based on an estimated head pose, determining three-dimensional ear positions, and processing audio signals to generate processed audio signals based on these ear positions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If presumptive dimensions such as average inter-ocular distance are used for scaling, then the tracking system can estimate ear positions, but the measurement precision deteriorates due to significant individual variations
Solution Approach 1:
The system uses the user's own facial landmarks (eyes, nose, mouth, jawline) as reference points for scaling, allowing each user to serve as their own reference standard. This eliminates the need for presumptive average dimensions and accommodates individual variations automatically.
Solution Approach 2:
The system dynamically adjusts the scaling parameters based on the detected facial landmark positions and head pose for each individual user, rather than using fixed average values. This allows the measurement precision to adapt to each user's unique anatomical dimensions.
2Device complexity
If a single standard imaging camera is used, then the device complexity is reduced, but the measurement precision of three-dimensional ear positions deteriorates
Solution Approach 1:
The system recovers three-dimensional ear position information by mathematically inferring depth from two-dimensional facial landmark positions and head pose angles. This allows a single 2D camera to provide 3D measurement capability through computational geometry.
Solution Approach 2:
The system uses facial landmarks as intermediary reference points that can be reliably detected by a single camera. These landmarks serve as mediators to infer the position of ears, which are not directly visible to the camera.
3Adaptability or versatility
If camera-based head tracking is implemented, then the audio system can adapt to user position, but the reliability deteriorates due to unacceptable noise and crosstalk cancellation
Solution Approach 1:
The system replaces direct physical measurement mechanisms with optical camera-based detection and computational processing. This allows for non-contact, dynamic tracking of head and ear positions to improve audio adaptation while maintaining reliability through accurate mathematical modeling.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In various embodiments, a computer-implemented method for audio processing based on a head pose of a user comprises acquiring one or more images of a user, processing the one or more images to identify a plurality of face landmarks representing locations on a head of the user, selecting, from the plurality of face landmarks and based on an estimated head pose of the user, a set of one or more landmark pairs, determine, based on the set of landmark pairs, three-dimensional positions of ears of the user; and processing one or more audio signals to generate one or more processed audio signals based on the three-dimensional positions of the ears.