Audio Occlusion Compensation in Augmented Reality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality systems fail to provide a realistic and immersive audio experience due to the occlusion of real audio sources by virtual objects, as they do not effectively account for the acoustic effects of virtual objects on real audio signals, leading to an incomplete and distorted sound perception.
Innovation Solution
An apparatus and method that determine the position of virtual objects relative to real audio sources, identify occluded regions, and process audio signals to remove or modify those occluded by virtual objects, ensuring that only non-occluded audio signals are heard, thereby enhancing the immersive experience by compensating for the acoustic effects of virtual objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If virtual objects are displayed in the augmented reality environment, then the visual immersion is improved, but the audio realism deteriorates due to occlusion effects from virtual objects blocking real audio sources
Solution Approach 1:
The patent converts the harmful occlusion effect into a beneficial feature by detecting when virtual objects block real audio sources and using this information to generate appropriate audio occlusion effects. The system processes audio signals to simulate how real objects would occlude sound, thereby restoring audio realism that would otherwise be lost due to the virtual object's visual presence.
Solution Approach 2:
The patent introduces an audio processing intermediary that acts as a mediator between the virtual object's visual occlusion and the real audio source. This intermediary processes the audio signal by analyzing the occluded region and applying appropriate filtering or attenuation to simulate the acoustic effect of the virtual object, thereby reconciling the visual and audio experiences.
2Loss of information
If audio signals are processed to remove occluded components, then the audio realism is improved, but the processing complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-defining occluded regions based on the virtual object's position and characteristics before audio processing occurs. The system pre-calculates which audio sources should be occluded and by how much, based on the virtual object's spatial parameters, thereby simplifying the real-time audio processing required during runtime.
Solution Approach 2:
The patent applies local quality by processing only the specific audio components that are occluded by virtual objects, rather than processing all audio signals uniformly. The system identifies occluded regions in space and applies occlusion effects only to audio sources within those regions, leaving other audio signals unchanged, thereby reducing overall processing complexity.
3Measurement precision
If the system processes all audio signals to account for occlusion, then the audio accuracy is improved, but the computational load increases
Solution Approach 1:
The patent applies partial action by processing only the necessary portion of audio signals - specifically those from sources located in occluded regions. The system determines which audio sources require occlusion processing based on their spatial relationship to virtual objects, and applies processing only to those specific sources, rather than processing all audio signals excessively.
Solution Approach 2:
The patent applies parameter changes by dynamically adjusting audio signal parameters (such as gain, frequency response, or spatial positioning) based on the degree of occlusion. The system modifies audio parameters in proportion to how much a virtual object blocks the real audio source, thereby achieving accurate occlusion effects with efficient computational methods.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
An apparatus, method and computer program are disclosed. The apparatus may comprise means for determining, relative to one or more real audio sources in an environment, a relative position of a virtual object in the environment. The apparatus may also comprise means for determining, based on the relative position, an occluded region of the environment in which audio signals from the one or more real audio sources would be occluded by the virtual object. The apparatus may also comprise means for outputting, to a user in the environment, audio data, the outputting means configured to operate in a first mode and in a second mode. The first mode is an open mode, and substantially no processing is performed on audio signals received from the one or more real audio sources in the first mode. In the second mode the audio data is generated by processing audio signals received from the one or more real audio sources to at least partially remove those audio signals which would be occluded by the virtual object. The apparatus may also comprise means for switching the outputting means between the first mode and the second mode based on a location of the user relative to the occluded region.