Audio Localization in Augmented Reality Live Event Views
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies fail to provide an immersive and localized audio-visual experience for remote participants in live eSports events, lacking synchronization of real-world venue acoustics with virtual views, which affects the authenticity of the remote viewing experience.
Innovation Solution
A system that generates a three-dimensional live view of a real-world venue and synchronizes audio to reflect the acoustics of specific locations within the venue, streamed via a network to a head-mounted display for remote users, creating an augmented reality experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional audio broadcasting is used for live events, then audio can be transmitted to remote users, but the audio experience lacks location-specific acoustics and immersion
Solution Approach 1:
The patent applies local quality by capturing audio from multiple microphones positioned at different locations within the venue and selectively applying acoustics processing based on the user's selected virtual seating location. Each location has its own acoustic characteristics that are applied to create an authentic local listening experience, transforming generic broadcast audio into location-specific audio environments.
Solution Approach 2:
The system performs preliminary action by pre-capturing audio from multiple fixed positions throughout the venue during the live event and pre-processing acoustic characteristics for different locations. This allows the system to quickly switch between different acoustic environments based on user selection without real-time processing delays, improving both authenticity and efficiency.
2Measurement precision
If multiple audio sources are used to capture venue acoustics, then location-specific audio experience is improved, but system complexity and processing requirements increase
Solution Approach 1:
The audio system is segmented into multiple independent microphone sources positioned at different locations, each capturing acoustic characteristics from its specific position. This segmentation allows the system to treat each audio source as a separate channel that can be independently processed and mixed, improving localization accuracy while managing complexity through modular processing.
Solution Approach 2:
The system introduces an intermediary audio processing layer that receives signals from multiple microphone sources and combines them according to the selected virtual seating location. This intermediary processing stage acts as a mediator between the raw multi-source audio inputs and the final localized audio output, enabling precise audio localization without requiring direct complex routing between all sources.
3Reliability
If 3D live view is generated for specific physical POV, then immersive visual experience is achieved, but synchronization with localized audio becomes more challenging
Solution Approach 1:
The system merges the audio and video processing pipelines by linking the selected physical POV for video generation with the corresponding audio source and acoustic processing parameters. When a user selects a specific seating location, both the visual viewpoint and audio perspective are simultaneously adjusted to match that location, ensuring synchronized immersive experience without temporal delays.
Data Source
AI summary
A method for localizing sound when enabling participation in a live event, including establishing a multi-player gaming session of a gaming application that generates an interactive gaming world, the live event being a real-world venue where players playing the gaming application are present. A live view of the venue is generated based on captured video streams, and generated for a physical POV anchored to a physical location in the venue. Audio is generated for the live event, and modified to reflect acoustics of the real-world venue for the physical location. The modified audio is synchronized to the live view. The live view and synchronized audio are streamed to an HMD of a remote user located outside the venue, and presents an augmented reality view of the live event.