AR Sound Source Identification via Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented and mixed reality systems struggle to distinguish and notify users of sounds originating from the real space, leading to user confusion and potential missed sounds.
Innovation Solution
An information processing apparatus that acquires user and virtual object information, along with sound source position information, to determine a notification method for directing the user to the sound source in the real space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all sounds in the real space are notified to the user, then the user can be aware of all real-space sounds, but the user feels troublesome and experiences confusion between real-space and virtual-space sounds
Solution Approach 1:
The patent segments sound sources into two distinct categories: real-space sounds and virtual-space sounds. By implementing separate processing paths for each type of sound, the system can notify users of real-space sounds through appropriate channels (such as visual indicators or spatial audio cues) while maintaining clear distinction from virtual-space sounds, thereby improving identification accuracy without causing user confusion or discomfort
Solution Approach 2:
The patent introduces an intermediary processing layer that analyzes sound sources and determines their origin (real-space or virtual-space) before notification. This intermediary layer acts as a mediator that filters and categorizes sounds, enabling selective notification of real-space sounds while preventing confusion with virtual-space sounds, thus resolving the contradiction between comprehensive sound awareness and user comfort
2Adaptability or versatility
If sounds in a mixed space are also heard, then the user experiences both real and virtual sounds, but it becomes difficult to judge whether a sound is from the real space or the mixed space
Solution Approach 1:
The patent applies the concept of visual differentiation (analogous to color changes) by associating distinct visual indicators with sounds from different spaces. Real-space sounds are accompanied by specific visual cues (such as icons or color-coded indicators) that differentiate them from virtual-space sounds, enabling users to easily identify sound origins even in a mixed reality environment where both types of sounds are present
Solution Approach 2:
The patent introduces an intermediary analysis mechanism that examines each sound source to determine its origin before presenting it to the user. This intermediary process preserves sound origin information by categorizing sounds as either real-space or virtual-space, preventing information loss and enabling users to distinguish between different sound sources in the mixed environment
3Reliability
If the user makes misjudgment about sound origin, then the user may miss important real-space sounds, but improving sound detection may increase system complexity
Solution Approach 1:
The patent segments the sound processing system into distinct functional modules: sound acquisition, sound source analysis, origin determination, and notification. By dividing the system into these specialized segments, each module can be optimized for its specific function, improving overall reliability in sound origin judgment while keeping individual module complexities manageable and maintainable
Data Source
AI summary
An information processing apparatus capable of more positively grasping a sound in a real space. User information concerning a user who visually recognizes a space image including at least an image of a virtual space is acquired. Virtual object information concerning a virtual object in the space image is acquired. In a case where a sound is generated in a real space, position information of a sound source of the generated sound is acquired. A notification method of notifying the user of a direction of the sound source in the real space is determined based on the acquired user information, the acquired virtual object information, and the acquired position information.