Wearable Device Spatial Audio Processing for Environmental Sound Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users wearing head-mounted display (HMD) devices for Virtual Reality (VR) or Augmented Reality (AR) experience difficulty in detecting surrounding environmental sounds due to the occlusion of their vision and hearing by the devices, making it hard to notice important sounds like footsteps or notifications.
Innovation Solution
A wearable electronic device equipped with a microphone array comprising at least two microphones, a control circuit, and stereo speakers, which calculates the directional angle and distance of environmental sounds using sound samples from the microphones and processes them to simulate the sound as if it is coming from the correct direction and distance, allowing the user to perceive environmental sounds while immersed in a virtual reality scene.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If users wear HMD devices with headphones or earphones to experience VR or AR, then the virtual reality experience is enhanced, but the ability to detect environmental sounds is reduced
Solution Approach 1:
The patent uses microphones as intermediary devices to capture environmental sounds that would otherwise be blocked by headphones. The captured sounds are then processed and transmitted to the user, serving as a mediator between the blocked sound source and the user's ear, enabling environmental awareness while maintaining VR/AR immersion.
Solution Approach 2:
The patent replaces the mechanical blocking of sound by headphones with an electronic solution. Instead of physically allowing sound to reach the ear (mechanical approach), the system uses microphones, processors, and speakers to electronically capture, process, and reproduce environmental sounds, substituting the mechanical sound transmission path with an electronic one.
2Adaptability or versatility
If headphones or earphones are used to block external sounds for VR immersion, then the virtual reality immersion is improved, but the awareness of important sounds like footsteps or notifications is lost
Solution Approach 1:
The patent extracts important sound information from the environmental soundscape by using microphones to capture and processors to identify significant sounds such as footsteps, notifications, or warnings. This extracted information is then selectively presented to the user, separating crucial auditory data from the general soundscape that is blocked by headphones.
Solution Approach 2:
The system provides feedback to the user about environmental sounds that would otherwise be missed. By continuously monitoring the acoustic environment and presenting relevant sound information back to the user through the HMD device, the system creates a feedback loop that maintains awareness of important sounds while preserving VR immersion.
3Difficulty of detecting and measuring
If the microphone array and sound processing system are added to the wearable device, then the environmental sound perception capability is improved, but the device complexity increases
Solution Approach 1:
The patent integrates multiple functions into the HMD device to reduce overall complexity. The same processors used for virtual reality rendering are also utilized for analyzing and processing environmental sounds. The speaker system serves dual purposes by outputting both virtual reality audio and environmental sound notifications, making the device multi-functional rather than adding dedicated separate systems.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables users to perceive environmental sounds in a way that mimics real-world hearing, allowing them to be aware of important sounds like footsteps or notifications while maintaining the virtual reality experience, and can even modify these sounds to fit the virtual reality scene, enhancing the user's awareness and immersion.
Implementation Method 1
a microphone array (120) configured for sensing an environmental sound (ES1) in the real scene
Implementation Method 2
analyzed the phase angle of the coherence between at least two microphones
Implementation Method 3
stereo speakers (140)...processes them to simulate the sound as if it is coming from the correct direction and distance
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A wearable electronic device includes a microphone array, stereo speakers and a control circuit. The microphone array is configured for sensing an environmental sound. The control circuit is coupled to the microphone array and the speaker. The control circuit is operable to calculate a distance and a directional angle from a source of the environmental sound to the wearable electronic device according to the environmental sound sensed by the microphone array, to generate an effect sound corresponding to the environmental sound, to process the effect sound for simulating the effect sound to be originated from a virtual location with the distance and the directional angle relative to the wearable electronic device, and to broadcast the processed effect sound through the pair of speakers.