Virtual Agent Mediating Ambient Sound Detection in Immersive Environments

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Users in immersive VR, AR, or MR environments often fail to notice ambient sounds due to their focus on the virtual world, which can lead to missed important sounds like fire alarms, baby cries, or doorbells, as the virtual environment occludes real-world auditory information.

Innovation Solution

The system receives audio and sensor inputs from the environment, analyzes them to identify parameters, and adjusts the virtual environment or audio outputs based on predetermined policies to prioritize and enhance the detection of ambient sounds, ensuring users can hear and notice critical sounds.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If the immersive environment generates sounds and images that occlude real-world auditory information, then the user's immersion in the virtual environment is improved, but the user's ability to detect ambient sounds deteriorates

Engineering Contradiction:

Improveimmersion qualityVSAvoidambient sound detection

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The system introduces a virtual agent as an intermediary that mediates between the immersive environment and the user. The virtual agent monitors ambient sounds and communicates them to the user, allowing the immersive environment to maintain its occluding sounds while still enabling ambient sound detection through the agent's announcements and visual cues

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system changes parameters of the virtual agent dynamically based on ambient sound analysis. When important ambient sounds are detected, the agent's volume, position, or visual prominence is adjusted to draw user attention without completely disrupting the immersive environment's audio-visual output

Inventive Principle:

Principle #35Parameter changes

2Productivity

If the virtual environment provides a complete sensory experience, then user engagement is improved, but user awareness of the real world deteriorates

Engineering Contradiction:

Improveuser engagementVSAvoidreal-world awareness

Core Design Contradiction:

ProductivityVSLoss of information

Solution Approach 1:

The system implements feedback by having the virtual agent continuously monitor ambient sounds and provide information about the real world to the user. This feedback loop maintains user engagement with the virtual environment while simultaneously restoring awareness of real-world events through the agent's communications

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The audio environment is segmented into virtual sounds and ambient sounds. The system selectively manages these segments, allowing virtual sounds to maintain immersion while using the virtual agent to selectively highlight important ambient sounds, thus preserving real-world awareness without compromising overall engagement

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS10154360B2Method and system of improving detection of environmental sounds in an immersive environment

Publication Date: 2018.12.11 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

An improved method and system for enhancing a user's experience within an immersive environment is disclosed. Solutions and implementations provided improve the user's ability to both hear and identify sounds from outside the user's immersive experience. Implementations include receiving audio and sensor input from the environment, analyzing the received input and adjusting either the display or audio outputs generated for the user based in part on pre-determined policies.