Mixed Reality Audio Control for External and Virtual Sound Balance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing sound output devices with noise cancellation functions in VR applications fail to provide audible external sound in MR applications unless talking behavior is detected, limiting user interaction with the real world.
Innovation Solution
An information processing apparatus that adjusts relative sound levels of external and virtual sound based on real and virtual images, enabling seamless integration of external and virtual sound in mixed reality environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If noise cancelling function is activated to cut off external sound in VR applications, then immersion in virtual reality is improved, but ability to hear external sound in mixed reality applications deteriorates
Solution Approach 1:
The noise cancelling function is made dynamically adjustable rather than fixed. The system automatically changes the noise cancelling level based on detected user actions (eye blinking, head movement) or image analysis (detecting real-world objects in the displayed image), allowing the device to switch between VR immersion mode and MR awareness mode as needed
Solution Approach 2:
The system incorporates feedback mechanisms by detecting user physiological signals (eye blinking, head movement) and environmental information (image content) to automatically adjust the noise cancelling level. This closed-loop control allows the system to respond to user needs and adapt the audio output accordingly
2Reliability
If noise cancelling is used to block external sound, then virtual reality experience is enhanced, but user interaction with real world deteriorates
Solution Approach 1:
The system performs preliminary detection of user actions (eye blinking, head movement) or image content before adjusting the noise cancelling level. This allows the system to proactively prepare for potential real-world interaction needs by preemptively reducing noise cancelling when real-world objects are detected or user interaction gestures are anticipated
Solution Approach 2:
The system automatically adjusts noise cancelling levels based on detected user actions or image analysis without requiring manual user intervention. The device serves itself by interpreting user physiological signals and environmental context to make appropriate audio adjustments
3Ease of operation
If external sound is made audible only when talking behavior is detected, then response to surrounding people is improved, but audibility of external sound in other situations deteriorates
Solution Approach 1:
The system dynamically adjusts noise cancelling based on multiple detection criteria beyond just talking behavior, including eye blinking, head movement, and image content analysis. This expands the situations in which external sound becomes audible, making the system more versatile while maintaining ease of operation
Data Source
AI summary
An information processing apparatus according to the present disclosure obtains a real image of a real space, a virtual image, external sound in the real space, and virtual sound. The information processing apparatus then adjusts relative sound levels of the virtual sound and the external sound based on at least one of the obtained images.