Adaptive Video Transcoder for WebRTC Bandwidth Heterogeneity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-party WebRTC videoconferences, central servers using the selective forwarding unit (SFU) model struggle to provide video streams that meet the display resolution and network bandwidth criteria of all participant devices, leading to suboptimal quality of experience (QoE) due to heterogeneous device resolutions and network conditions.
Innovation Solution
Implementing a media server with a real-time adaptive video transcoder that determines heterogeneity in available network bandwidths and CPU/memory resources to perform adaptive video transcoding of large video streams into smaller streams, which are then selectively forwarded to devices with low bandwidth, improving QoE across all participants.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the central server requests participant devices to reduce video encoding bitrate to accommodate devices with lowest available network bandwidth, then network bandwidth utilization is improved, but quality of experience for participant devices with higher bandwidth capability deteriorates
Solution Approach 1:
The patent segments the video stream into multiple versions with different quality levels (original high-quality stream and transcoded low-quality stream). The SFU divides the incoming video stream and creates multiple output streams tailored to different device capabilities, allowing simultaneous delivery of high-quality streams to devices with high bandwidth and low-quality streams to devices with low bandwidth.
Solution Approach 2:
The patent changes the parameters of the video stream by performing real-time transcoding to convert the original video stream into a lower-quality version. The transcoder modifies bitrate, resolution, and other encoding parameters to create a stream suitable for devices with limited network bandwidth, while the original stream remains available for high-capability devices.
2Manufacturing precision
If the central server stops sending video streams to participant devices with low available network bandwidth to provide high quality streams to other devices, then quality of experience for high bandwidth devices is improved, but quality of experience for low bandwidth devices deteriorates
Solution Approach 1:
The SFU segments the video distribution by creating separate delivery paths: one for the original high-quality stream to high-capability devices and another for the transcoded low-quality stream to low-capability devices. This segmentation ensures that high bandwidth devices receive optimal quality without being constrained by the needs of low bandwidth devices.
Solution Approach 2:
The transcoder acts as an intermediary component between the SFU and low-bandwidth devices. It receives the original video stream from the SFU, performs real-time transcoding to adapt it for low-bandwidth conditions, and delivers the adapted stream to devices that cannot handle the original high-quality stream.
3Adaptability or versatility
If the central server performs real-time adaptive video transcoding on large video streams to produce small video streams for devices with low available bandwidth, then quality of experience for low bandwidth devices is improved, but device complexity and processing resources increase
Solution Approach 1:
The transcoder serves as an intermediary that offloads the complex transcoding processing from the SFU and participant devices. It is a dedicated component with specialized hardware or software optimized for real-time video encoding/decoding operations, reducing the processing burden on other system components.
Solution Approach 2:
The transcoded video stream is generated autonomously by the transcoder based on real-time assessment of device capabilities and network conditions. The SFU automatically directs appropriate streams to appropriate devices without manual intervention, and the transcoder continuously adapts stream quality based on feedback from devices and network conditions.
4Adaptability or versatility
If the central server performs real-time adaptive video transcoding considering heterogeneity of display resolutions and network bandwidths, then adaptability to heterogeneous devices is improved, but device complexity and processing overhead increase
Solution Approach 1:
The SFU applies local quality adaptation by delivering different quality levels of video streams to different devices based on their specific capabilities. Each device receives a video stream optimized for its display resolution and network bandwidth, rather than all devices receiving the same stream. This localized optimization minimizes unnecessary processing while maximizing quality for each device.
Solution Approach 2:
The video streaming system dynamically adapts to changing device capabilities and network conditions. The SFU continuously monitors device information and network state, and the transcoder adjusts encoding parameters in real-time based on current conditions, allowing the system to respond dynamically to heterogeneity without requiring static pre-configuration.
Data Source
AI summary
Techniques for providing selective video forwarding in real-time video applications involving multiple user devices. The techniques can be employed in WebRTC videoconferences, taking into account the heterogeneous natures of device display resolutions and networks in which the respective devices operate. The techniques employ a selective forwarding unit (SFU) having a real-time adaptive video transcoder. The SFU determines whether a benefit to quality of experience (QoE) can be achieved by performing, in real-time, adaptive video transcoding on large video streams produced by the respective devices. If a heterogeneity value associated with available network bandwidths is greater than a predetermined value and sufficient CPU/memory resources are available, then the real-time adaptive video transcoder performs, in real-time, adaptive video transcoding on the large video streams to produce corresponding small video streams, which are selectively forwarded to user devices having low available network bandwidths.