Physics-Based Audio Synthesis for Mixed-Reality Collision Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing XR systems fail to present virtual audio that accurately reflects a user's surroundings, leading to auditory inconsistencies and discomfort, such as motion sickness, due to the lack of consideration for real-world acoustic properties in synthesized sounds.
Innovation Solution
A method for generating virtual audio in mixed reality systems that determines if an audio model exists for a virtual object, and if not, creates a custom audio model based on the object's acoustic properties, synthesizing an audio signal that reflects the collision and presenting it via a head-wearable device.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If virtual audio is synthesized without considering real-world acoustic properties, then the system complexity is reduced and audio generation is faster, but auditory consistency and user immersion deteriorate
Solution Approach 1:
The system changes acoustic parameters (reverberation time, early decay time, clarity) based on detected environmental characteristics to dynamically adjust virtual audio synthesis, achieving both speed and consistency through parameter-based adaptation rather than complete physical modeling
2Reliability
If a database of real sounds is used to synthesize virtual sounds, then auditory realism is improved, but memory requirements and system complexity increase
Solution Approach 1:
The system replaces the mechanical approach of storing and mixing recorded sound waves with a physics-based acoustic model that calculates sound propagation through environmental parameters, reducing memory needs while maintaining realism
Solution Approach 2:
Instead of storing complete sound databases, the system stores acoustic parameters (reverberation time, early decay time, clarity) and calculates virtual sounds on-demand by adjusting these parameters, significantly reducing memory requirements
3Device complexity
If virtual audio does not account for user surroundings, then system simplicity is maintained, but user comfort and immersion deteriorate
Solution Approach 1:
The system uses environmental sensors to detect acoustic characteristics of the real world and feeds this information back into the audio synthesis process, dynamically adjusting virtual audio to match the physical environment and prevent sensory conflict that causes motion sickness
Data Source
AI summary
Disclosed herein are systems and methods for generating and presenting virtual audio for mixed reality systems. A method may include determining a collision between a first object and a second object, wherein the first object comprises a first virtual object. A memory storing one or more audio models can be accessed. It can be determined if the one or more audio models stored in the memory comprises an audio model corresponding to the first object. In accordance with a determination that the one or more audio models comprises an audio model corresponding to the first object, an audio signal can be synthesized, wherein the audio signal is based on the collision and the audio model corresponding to the first object, and the audio signal can be presented to a user via a speaker of a head-wearable device. In accordance with a determination that the one or more audio models does not comprise an audio model corresponding to the first object, an acoustic property of the first object can be determined, a custom audio model based on the acoustic property of the first object can be generated, an audio signal can be synthesized, wherein the audio signal is based on the collision and the custom audio model, and the audio signal can be presented, via a speaker of a head-wearable device, to a user.