Mixed Reality Audio Analysis for Sound Source Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current mixed reality systems lack effective methods to analyze and present audio data to determine the location and classification of sound sources, limiting their ability to provide immersive and informative experiences in augmented reality environments.
Innovation Solution
A computer-implemented method that receives audio data from multiple microphones, analyzes it to determine the location and classification of sound sources, and presents graphic elements on a near-to-eye display indicating the source's location and movement, using filtering, machine learning models, and sensory data integration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If audio data analysis is implemented to determine sound source location and classification, then situational awareness and information quality are improved, but device complexity and processing requirements increase
Solution Approach 1:
The audio data analysis is divided into separate functional modules: audio data reception module, analysis module for determining location and classification, and graphic element presentation module. This segmentation allows each module to be optimized independently and simplifies the overall system architecture.
Solution Approach 2:
Graphic elements serve as an intermediary representation of audio data analysis results. Instead of directly processing and displaying raw audio data, the system uses graphical indicators to convey sound source location and classification information, reducing processing complexity while maintaining information quality.
2Measurement precision
If multiple audio input devices are used to improve sound source localization accuracy, then measurement precision is improved, but device complexity increases
Solution Approach 1:
Multiple audio input devices (microphones) are combined into a unified audio analysis system. The audio data from multiple microphones is processed together to determine sound source location and classification, achieving improved measurement precision through integrated processing rather than separate analysis of each microphone.
Data Source
AI summary
Disclosed herein is a mixed reality application to use a multi-channel audio input to identify a character and origin of a given sound, then present a visual representation of the given sound on a near eye display. The visual representation including a vector to the source of the sound. The visual representation further including graphical elements that describe various attributes of the given sound including the magnitude, directionality, source, and threat level. Where the source of the given sound is moving, the visual representation shifts to illustrate the movement.