Game Streaming Spatial Audio Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing audio streaming technologies are limited in their ability to adapt to diverse audio equipment and spatialization technologies, often failing to provide optimal spatial audio experiences when content is streamed from one device to another with different capabilities.
Innovation Solution
The generation and streaming of sound-field data independent of speaker configuration or spatialization technology, allowing recipients to utilize the best available spatialization technologies, with the audio platform module converting and transcoding audio data to match local and remote device capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If audio is generated in a format directly compatible with the player's headphones (e.g., two-channel audio), then the player's local audio experience is optimized, but the streamed content cannot utilize better spatialization technologies available at remote devices
Solution Approach 1:
The patent segments the audio processing into two distinct stages: (1) sound-field data generation at the source device using bank coefficients and monaural sources, and (2) format conversion at the remote device based on available spatialization technologies. This segmentation allows the source to generate universal sound-field data while enabling remote devices to optimize for their specific hardware capabilities.
Solution Approach 2:
The patent introduces sound-field data as an intermediary representation that bridges the source device and various remote audio reproduction systems. This intermediary format contains embedded directional information that can be transcoded to multiple spatialization formats (DOLBY ATMOS, DTS:X, etc.), acting as a universal adapter between different audio ecosystems.
2Adaptability or versatility
If sound-field data is generated independently of speaker configuration, then the audio can be streamed to diverse devices with different spatialization capabilities, but the processing required to convert to device-specific formats increases complexity
Solution Approach 1:
The patent applies preliminary action by pre-calculating and embedding bank coefficients and directional information into the sound-field data at the source device. This preliminary processing establishes a robust foundation that enables efficient format conversion at the remote device, as the essential spatial information is already encoded and ready for transcoding to various spatialization formats.
3Manufacturing precision
If the game engine generates audio data optimized for the player's local device, then local audio quality is maximized, but the streamed content loses the opportunity to utilize superior spatialization technologies at the recipient's device
Solution Approach 1:
The patent applies dynamics by making the audio format adaptive rather than static. The sound-field data maintains precise spatial information from the source while enabling dynamic format conversion at the remote device based on available spatialization technologies. This allows the audio to be precisely generated at the source and then dynamically adapted to utilize superior technologies at the destination.
Data Source
AI summary
A game engine may generate video and audio content on a per-frame basis. Audio data corresponding to a current frame may be generated to comprise sound-field information independent of a speaker configuration or spatialization technology that may be used to play the associated audio. The sound-field may be generated based on monaural audio data corresponding to a sound produced by an in-game object at the object's position as of the current frame. The sound-field information may be transmitted to a remote computing device for reproduction using a selected, available speaker configuration and spatialization technology.