Dynamic Speech Directivity Reproduction in Artificial Reality

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional telepresence systems fail to accurately reproduce dynamic speech directivity, leading to inconsistencies in sound perception for listeners in artificial reality environments, as they do not account for the speaker's pose and environment-specific reverberant properties.

Innovation Solution

An artificial reality system captures voice input and pose data to create a directivity-attuned voice signal that simulates the speaker's dynamic speech directivity within the artificial reality environment, using processing modules to adjust sound propagation based on the speaker's pose and relative position to the listener, and the environmental reverberant properties.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If conventional telepresence systems replay captured user speech without accounting for dynamic speech directivity, then the system is simple to implement, but the sound perception consistency and realism deteriorate

Engineering Contradiction:

Improvesystem implementation simplicityVSAvoidsound perception consistency

Core Design Contradiction:

Ease of manufactureVSReliability

Solution Approach 1:

The patent applies dynamics by making the speech directivity pattern adaptive and time-varying based on the speaker's pose and speech content. Instead of using a fixed directivity pattern, the system dynamically adjusts the pattern to match the speaker's head orientation, body pose, and vocal characteristics in real-time, thereby achieving consistent sound perception while maintaining reasonable system complexity

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes multiple parameters of the directivity pattern including orientation, shape, and temporal characteristics to match the speaker's real-time pose and speech characteristics. This parameter adaptation allows the system to reproduce dynamic speech directivity accurately without requiring complex hardware modifications

Inventive Principle:

Principle #35Parameter changes

2Device complexity

If the system does not account for speaker pose and environmental reverberant properties, then the processing complexity is low, but the authenticity and immersion of telepresence experience deteriorate

Engineering Contradiction:

Improvesignal processing complexityVSAvoidenvironmental adaptability

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

The system performs preliminary actions by capturing the speaker's pose data and environmental reverberant properties before processing the speech signal. This preliminary capture of spatial and acoustic context allows the system to pre-compute appropriate directivity patterns and reverberation characteristics, reducing real-time processing complexity while maintaining high environmental adaptability

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces intermediary processing modules that separately handle pose data, reverberant properties, and speech signal processing. These intermediary components act as mediators between the captured data and the final audio output, enabling the system to account for multiple factors (pose, environment, speech content) without creating a single complex processing bottleneck

Inventive Principle:

Principle #24Intermediary (Mediator)

3Measurement precision

If additional hardware is used to capture dynamic speech directivity, then the measurement precision improves, but the device complexity and cost increase

Engineering Contradiction:

Improvedirectivity capture accuracyVSAvoidhardware requirements

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system replaces complex mechanical directivity measurement hardware with computational methods. Instead of using multiple microphones arranged in specific geometric patterns or specialized directivity measurement equipment, the patent uses pose data from sensors (cameras, IMUs) and environmental acoustic measurements to computationally derive and synthesize the directivity pattern, achieving high measurement precision with simpler hardware

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS11758347B1Dynamic speech directivity reproduction

Publication Date: 2023.09.12 META PLATFORMS TECHNOLOGIES LLC

AI summary

The disclosed computer-implemented method may include capturing, via a headset microphone of a speaker's artificial reality device, voice input of a speaker in communication with a listener in an artificial reality environment. The method may include detecting a pose of the speaker within the artificial reality environment and determining a position of the speaker relative to a position of the listener within the artificial reality environment. The method may further include processing, based on the pose and the relative position of the speaker within the artificial reality environment, the voice input to create a directivity-attuned voice signal for the listener, and delivering the directivity-attuned voice signal to an artificial reality device of the listener. Various other methods, systems, and computer-readable media are also disclosed.