Spatialized Audio Streaming for Spectator Immersion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems fail to provide complete and real-time immersion for spectators attending sporting events remotely, as they do not effectively account for the spectator's position and orientation, leading to a lack of personalized and immersive audio experience.
Innovation Solution
A process that analyzes sound scenes by classifying audio flows based on sound sources, applying gains according to spectator choice and orientation, using deep neural networks for real-time classification and source separation, and incorporating an orientation sensor to determine the spectator's position and orientation, allowing for spatialized sound restitution and informative audio flow selection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple multimedia streams are transmitted to spectators, then they can access various sporting events, but complete spectator immersion is not achieved
Solution Approach 1:
The patent segments the multichannel audio stream into multiple single-channel streams, each corresponding to a specific sound source location. This allows the system to selectively process and transmit only the audio streams relevant to the spectator's position and orientation, thereby achieving immersion while maintaining access to multiple events through the same technical framework
Solution Approach 2:
The patent applies local quality by tailoring the audio stream content to each spectator's specific position and orientation. The system adjusts which audio channels are transmitted based on individual spectator characteristics, creating a personalized immersive experience for each user while the overall system maintains versatility for multiple events
2Reliability
If audio streams are filtered and delivered based on viewer position, then immersion is improved, but real-time complete immersion is not achieved
Solution Approach 1:
The patent applies preliminary action by pre-classifying audio streams into single-channel streams associated with specific sound source locations before transmission. The description table is prepared in advance, containing all necessary information about audio stream locations and characteristics, enabling rapid real-time processing at the terminal without compromising immersion quality
Solution Approach 2:
The patent implements dynamics by enabling the terminal to dynamically select and process audio streams based on the spectator's real-time position and orientation. The system adapts the audio output in real-time by adjusting which pre-classified streams are activated and how they are spatialized, achieving both immersion and real-time responsiveness
3Adaptability or versatility
If spatialized sound reproduction is applied, then personalized audio experience is achieved, but system complexity increases
Solution Approach 1:
The patent applies copying by creating a virtual representation of the acoustic environment through the description table, which contains copied metadata about sound source locations and audio stream characteristics. This virtual model allows the terminal to reproduce spatialized sound without requiring complex physical acoustic processing, simplifying the actual device implementation while maintaining personalized audio experience
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The invention relates to a method for broadcasting a spatialised audio stream to terminals (TERM) of spectators attending a sporting event. The method comprises acquiring a plurality of audio streams which constitute a sound scene (SS). The sound scene (SS) is analysed by a server (SERV) in order to make an assessment of the sound spatialisation of the audio streams and the reproduction thereof on the terminals (TERM), depending, on the one hand, on the location of the audio streams and, on the other hand, on the position of the spectators.