Free-Viewpoint Audio Channel Prioritization for Low-Latency Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies struggle to deliver high-quality, low-latency free viewpoint audio over conventional IP networks to consumer devices with limited audio processing capabilities, due to network congestion and latency issues, which are not addressed by professional-grade equipment suitable for prosumer or consumer applications.
Innovation Solution
A method for generating a coded monitor mix that prioritizes and encodes audio channels based on user preferences, device capabilities, and network conditions, using a channel selection and prioritization scheme to optimize the audio delivery over wireless IP networks, ensuring high quality and low latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If all audio channels are transmitted individually over wireless IP network, then complete audio information is preserved, but network congestion and latency increase
Solution Approach 1:
The patent segments the 66 audio channels into priority groups (first priority: close-up microphones, second priority: OZO arrays) and transmits them separately. This allows critical audio information to be delivered first with lower latency, while less critical channels follow, resolving the contradiction between complete information transmission and latency reduction.
Solution Approach 2:
The patent extracts and prioritizes essential audio channels (close-up microphones capturing primary sound sources) from the complete set of 66 channels. By separating critical from non-critical audio data, the system transmits only essential information first, reducing latency while preserving the most important audio content.
2Loss of information
If all 66 audio channels are processed and transmitted, then complete audio coverage is achieved, but consumer devices with limited processing capabilities cannot handle the content
Solution Approach 1:
The patent extracts only the essential audio channels needed for high-quality monitoring from the complete 66-channel set. By identifying and transmitting only critical channels (close-up microphones and selected OZO arrays), the system reduces processing complexity for consumer devices while preserving the most important audio information.
Solution Approach 2:
The patent applies different quality levels to different audio channels based on their importance. High-priority channels (close-up microphones) receive full processing and transmission, while lower-priority channels are reduced or omitted. This local differentiation of quality allows consumer devices to handle the content effectively while maintaining high audio coverage for critical sources.
3Loss of information
If professional-grade equipment is used for audio over IP networks, then lossless audio transmission is achieved, but expensive infrastructure is required
Solution Approach 1:
The patent extracts only the essential audio channels that provide high-quality monitoring experience, eliminating the need to transmit all 66 channels in lossless format. This extraction approach allows prosumer and consumer devices to achieve excellent audio quality for critical sources without requiring expensive professional infrastructure for complete lossless transmission.
Solution Approach 2:
The patent enables the use of inexpensive consumer devices (smartphones, tablets, consumer headsets) instead of expensive professional audio equipment. By prioritizing and transmitting only essential audio channels, the system makes high-quality audio monitoring accessible on affordable, widely available consumer hardware, eliminating the need for costly professional-grade infrastructure.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An audio scene is captured in multiple audio channels by distributed microphones. Based on at least one of a viewpoint of a user and at least one preference of a user, two or more of those multiple audio channels are selected (400, 500) from among those multiple audio channels; those selected two or more audio channels are prioritized (500, 600) based on at least one of preferences of the user, requirements of a software application of a device (700) operated by the user, and requirements of operating conditions of the device (700). After that prioritizing (500, 600), at least one of the selected two or more audio channels is encoded (600) into a coded monitor mix (610-1) which is sent to the device (700) over a transmission link.