AR Audio Recognition for Visual Labeling of Off-Screen Sounds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) devices fail to intuitively inform users of events occurring near them using audio identification technology, particularly for sounds outside the visible area, and lack real-time detection and labeling of sound sources.
Innovation Solution
An AR device with a see-through display, audio inputter, and controller that identifies audio signals and controls the display to output corresponding virtual objects, combining audio and image information for enhanced user perception.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If audio identification technology is integrated into AR devices to detect sound sources, then the user's sensory area is expanded, but the device complexity increases
Solution Approach 1:
The patent combines audio identification technology with AR device functionality by integrating an audio inputter, controller with audio recognition algorithms, and see-through display into a single unified system. The controller receives audio signals, identifies sound sources and events, and displays corresponding virtual objects through the see-through display, merging multiple sensing and processing functions into one device.
Solution Approach 2:
The AR device is designed to perform multiple functions: it captures audio signals from the environment, identifies sound sources and events through audio recognition, displays visual information through the see-through display, and provides spatial awareness. This multi-functional design expands the device's versatility while managing complexity through integrated architecture.
2Speed
If the AR device processes audio signals in real time to identify events, then the response speed is improved, but the computational power required increases
Solution Approach 1:
The system performs preliminary audio signal processing and event identification in real-time as audio signals are received. The controller continuously monitors audio inputs and immediately identifies sound sources and events, preparing visual information for display before the user needs to see it, thereby reducing latency and improving response speed.
Solution Approach 2:
The patent replaces traditional mechanical or manual audio analysis methods with electronic digital signal processing and artificial intelligence-based audio recognition algorithms. This substitution enables real-time processing of audio signals with high accuracy, improving response speed while managing computational requirements through efficient algorithm implementation.
3Loss of information
If the AR device displays visual information for sounds outside the visible area, then the information completeness is improved, but the attention of the user is divided
Solution Approach 1:
The patent addresses the limitation of visual area by adding an audio dimension to the AR experience. The system detects sound sources from all directions around the user, not just within the visual field, and displays corresponding virtual objects that represent sounds outside the visible area. This multi-dimensional sensing approach provides complete environmental information without requiring the user to constantly scan their surroundings.
Solution Approach 2:
The see-through display acts as an intermediary that presents audio information in a visual format without requiring the user to shift attention between audio and visual channels. The controller processes audio signals and translates them into visual representations that can be perceived peripherally or directly, allowing the user to maintain focus while receiving complete environmental information.
Data Source
AI summary
Proposed is an augmented reality device capable of performing audio identification and a control method therefor. The augmented reality device comprises: a see-through display which is formed to enable a user's eyes to see therethrough and thus outputs a virtual object; an audio input unit which receives an input of an audio signal generated within a preset distance from the display; and a control unit which controls operations of the see-through display to identify event information corresponding to the audio signal and to output image information of the virtual object corresponding to the identified event information.