Bitrate Allocation for Immersive Multimedia Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing immersive multimedia streaming technologies allocate bitrates independently to video and audio signals without considering their synchronized content, leading to suboptimal quality of experience (QoE) due to reduced quality in one signal when enhancing another.
Innovation Solution
A method that produces video and audio saliency maps to identify content of interest, combines them into a fused saliency map, and adjusts bitrate allocations between video and audio signals based on the map to prioritize content that is most engaging, using relative weights and statistical data on user interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If bitrate is allocated independently to video and audio signals, then encoding complexity is reduced, but quality of experience deteriorates due to suboptimal bitrate distribution
Solution Approach 1:
The patent combines video and audio bitrate allocation into a unified process that considers both signals together. A joint bitrate allocation model is implemented that analyzes video and audio content characteristics simultaneously to determine optimal bitrate distribution, thereby improving quality of experience while maintaining manageable encoding complexity through integrated processing.
Solution Approach 2:
The patent dynamically adjusts bitrate allocation parameters based on content characteristics. By analyzing video complexity, audio quality requirements, and their interrelationships, the system changes bitrate parameters adaptively to optimize quality of experience without requiring overly complex encoding structures.
2Manufacturing precision
If bitrate is allocated to enhance one signal (e.g., video), then quality of that signal is improved, but quality of the other signal (e.g., audio) deteriorates
Solution Approach 1:
The patent applies local quality principles by allocating different bitrate portions to video and audio signals based on their specific content characteristics and quality requirements. Rather than uniform allocation, the system dynamically determines optimal bitrate distribution to each signal locally, ensuring that enhancing one signal does not excessively compromise the other through balanced, content-aware allocation.
Solution Approach 2:
The patent implements dynamic bitrate allocation that adjusts in real-time based on content analysis. The system continuously evaluates video and audio signal characteristics and dynamically redistributes bitrate to maintain optimal quality for both signals, preventing quality degradation in one signal when enhancing the other through adaptive, real-time adjustments.
3Reliability
If synchronized content analysis between video and audio is performed, then quality of experience is improved, but processing complexity increases
Solution Approach 1:
The patent performs preliminary analysis of video and audio content characteristics before final bitrate allocation. By pre-processing and identifying key content features, temporal patterns, and synchronization points in advance, the system reduces the complexity of real-time synchronized analysis while maintaining improved quality of experience through informed bitrate distribution decisions.
Data Source
AI summary
Signals of an immersive multimedia item are jointly considered for optimizing the quality of experience for the immersive multimedia item. During encoding, portions of available bitrate are allocated to the signals (e.g., a video signal and an audio signal) according to the overall contribution of those signals to the immersive experience for the immersive multimedia item. For example, in the spatial dimension, multimedia signals are processed to determine spatial regions of the immersive multimedia item to render using greater bitrate allocations, such as based on locations of audio content of interest, video content of interest, or both. In another example, in the temporal dimension, multimedia signals are processed in time intervals to adjust allocations of bitrate between the signals based on the relative importance of such signals during those time intervals. Other techniques for bitrate optimizations for immersive multimedia streaming are also described herein.