Avatar Voice Acoustic Processing for Real-Time Virtual Spaces
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In metaverse virtual spaces, the sense of reality is diminished when the acoustic effect is not applied in real time to the voice of a conversation partner, leading to a loss of the conversation partner's sense of existence.
Innovation Solution
An information processing device and method that acquires a user's voice and applies acoustic characteristics matching the environment of the scene or area based on colliders associated with the scene or area, ensuring appropriate acoustic effects are applied to the conversation partner's voice in real time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If acoustic effect is applied to environmental sound but not to conversation partner's voice, then environmental immersion is improved, but sense of reality is lost
Solution Approach 1:
The patent applies different acoustic characteristics to different sound sources based on their location and type. Environmental sounds receive acoustic effects matched to the scene's acoustic environment, while conversation partner voices receive acoustic effects matched to the listener's position and the scene's acoustic characteristics. This local differentiation ensures that both environmental immersion and sense of reality are maintained simultaneously.
Solution Approach 2:
The system dynamically adjusts acoustic characteristics in real-time based on the listener's position, the conversation partner's position, and the scene's acoustic environment. The acoustic characteristics application unit continuously adapts the acoustic effects to maintain consistency with the current virtual environment state, ensuring the sense of reality is preserved during movement and interaction.
2Reliability
If acoustic characteristics are applied to conversation partner's voice, then sense of reality is improved, but processing complexity increases
Solution Approach 1:
The system pre-determines acoustic characteristics for different scenes and positions before real-time processing. The acoustic environment determination unit analyzes the scene and listener position in advance to select appropriate acoustic characteristics, reducing the computational burden during real-time voice processing while maintaining high sense of reality.
Solution Approach 2:
The patent introduces an intermediary acoustic characteristics application unit that mediates between the raw voice signal and the final output. This intermediary component applies scene-appropriate acoustic characteristics as a processing layer, simplifying the overall system architecture by centralizing the acoustic effect application logic.
3Illumination intensity
If acoustic effects are updated in real time with user movement, then immersion is improved, but computational load increases
Solution Approach 1:
The system implements a feedback mechanism where the acoustic environment determination unit continuously monitors the listener's position and scene changes, adjusting acoustic characteristics accordingly. This feedback loop ensures acoustic immersion remains high while optimizing computational load by updating only when necessary based on position changes and scene transitions.
Data Source
AI summary
There is provided an information processing device, an information processing method, and a program that can provide a user experience with a further improved sense of reality. When a second avatar associated with a second user is present in a scene or a plurality of areas associated with a virtual space in which a first avatar associated with a first user is present, a voice acquisition unit acquires a voice of the second user, an acoustic environment determination processing unit performs acoustic environment determination processing of determining an acoustic environment of the scene or the areas in which the first avatar is present based on a collider associated with the scene or the areas, and an acoustic characteristics application unit applies acoustic characteristics matching a processing result of the acoustic environment determination processing to the voice of the second user. The present technology can be applied to, for example, a system that provides a metaverse virtual space.