Audio-Based Augmented Reality Spatialization for Visually Impaired Accessibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional augmented reality (AR) technologies are predominantly visual, making them difficult or impossible for visually impaired individuals to use and presenting challenges such as the need for undivided visual attention and the cost, fragility, and unwieldiness of head-mounted displays.
Innovation Solution
A computer-implemented method and system for providing audio-based AR, which uses machine learning models and audio spatialization to identify and classify objects in an environment, associating audio tags with sounds that appear to emanate from these objects, allowing users to interact through spoken questions and receive answers, and generating audio spatialization data to simulate sound propagation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional visual AR technologies are used, then AR functionality is provided, but accessibility for visually impaired users is poor
Solution Approach 1:
The patent replaces visual display mechanisms with acoustic output mechanisms. Instead of using screens, lenses, and visual interfaces, the system uses speakers, audio processing, and spatial sound positioning to convey AR information. This substitution makes the technology accessible to visually impaired users who cannot rely on visual displays.
Solution Approach 2:
The patent changes the sensory modality parameter from visual to acoustic. By transforming the output medium from light-based displays to sound-based audio cues, the system adapts to the needs of visually impaired users while maintaining the core AR functionality of overlaying digital information on the physical world.
2Adaptability or versatility
If head-mounted displays are used for AR, then immersive AR experience is provided, but cost, fragility, and unwieldiness increase
Solution Approach 1:
The patent extracts the audio processing and output components from the traditional head-mounted display system. By separating the acoustic AR functionality from the visual display hardware, the system eliminates the need for expensive, fragile optical components while maintaining immersive AR capabilities through spatial audio alone.
Solution Approach 2:
The patent employs standard audio output devices (speakers, headphones) that are inexpensive and widely available, replacing costly specialized display hardware. This approach uses common, durable components to achieve AR functionality without the high cost and fragility of head-mounted displays.
3Loss of information
If visual AR interfaces are used, then information is displayed, but undivided visual attention is required
Solution Approach 1:
The patent transitions the information delivery from the visual dimension to the auditory dimension. By using spatial audio cues that can be perceived without visual engagement, the system delivers AR information while allowing users to maintain visual attention on their surroundings or other tasks.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems, methods, and non-transitory computer readable media are configured to receive a user request to identify at least one object of an environment in which a computing device is situated. A classification for the at least one object can be received. Subsequently, an audio tag based on the classification for the at least one object can be placed in a representation of the environment. The audio tag can be associated with a sound perceived by a user to be emanating from the least one object.