VR Missed Event Content Buffer for Spatial Audio
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In virtual reality systems, users often miss visual events associated with directional audio due to the limited field of view, as they cannot see the source of sounds originating from outside their current view, leading to a disjointed experience.
Innovation Solution
An apparatus and method that extract and display 'missed event content' based on changes in the user's viewing direction, allowing users to see visual imagery related to audio events they heard but missed, by analyzing audio and video data to determine the temporal start point of the event and providing it in reverse playback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If spatial audio is provided in virtual reality content, then the user can hear events from all directions, but the user may hear events without being able to see them due to limited field of view
Solution Approach 1:
The system performs preliminary action by detecting audio events and preparing missed event content in advance. When an audio event is detected outside the current field of view, the system captures and stores the associated visual content, then retrieves and displays it upon user request, ensuring the visual information is available even though it was missed during the original viewing moment
Solution Approach 2:
The system introduces an intermediary mechanism - a missed event content buffer and retrieval system - that mediates between the audio input and visual output. This intermediary captures audio events, associates them with their visual context, and provides on-demand visual playback, bridging the gap between what the user hears and what they can see
2Ease of operation
If the user changes viewing direction to look at the source of an audio event, then the user can see the visual event, but by the time the user looks in the perceived direction, they have missed the corresponding video imagery
Solution Approach 1:
The system creates a copy of the missed visual content and stores it in a buffer. When the user turns their head toward an audio event, the system retrieves the copied visual content from the buffer and displays it, allowing the user to see what they missed without disrupting their natural head movement and navigation behavior
Solution Approach 2:
The system implements feedback by monitoring the user's viewing direction changes and providing relevant visual content in response. When the user's head movement indicates they are looking toward an audio event source, the system detects this change and provides the corresponding missed visual content as feedback, creating a closed loop between user action and system response
3Volume of moving object
If the virtual reality view provides a limited spatial portion of video imagery, then the headset remains compact and comfortable, but the user cannot see events occurring outside their current viewing direction
Solution Approach 1:
The system extracts the missed visual content from the original video stream and separates it into a dedicated missed event content buffer. This extracted content is then made available on-demand without requiring the headset to display it continuously, allowing the headset to maintain its compact form factor while still providing access to peripheral visual events when needed
Data Source
AI summary
An apparatus configured to: in respect of virtual reality content comprising video imagery and spatial audio wherein the perceived direction to an audio event related to a visual event in the video imagery corresponds to a visual location of said visual event as presented in the virtual reality space; based on a change in the viewing direction of the virtual reality view by the user from a first viewing direction to a second viewing direction, and wherein said second viewing direction corresponds to, within a direction threshold, a direction of a particular audio event; providing for display of missed event content thereby allowing a user to see a missed visual event for which they heard the associated audio event and reacted thereto by changing the viewing direction.