Device-Adaptive Super-Resolution for Adaptive Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile video streaming faces challenges in maintaining optimal Quality of Experience (QoE) due to video quality fluctuations caused by throughput oscillations in heterogeneous environments, with conventional adaptive bitrate algorithms consuming excessive bandwidth and requiring high computational power for super-resolution enhancements.
Innovation Solution
A device-adaptive super-resolution based approach for adaptive streaming, which employs a super-resolution-based adaptive bitrate algorithm that computes costs for requesting video representations based on bandwidth, buffer status, and power consumption, using a weighted cost function to select optimal representations and apply super-resolution networks to enhance video quality while minimizing data usage and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional ABR algorithms select high bitrate segments to improve video quality, then video quality is improved, but bandwidth consumption increases and network congestion occurs
Solution Approach 1:
The patent replaces the conventional mechanical approach of transmitting high-bitrate video data over the network with a computational approach using super-resolution neural networks. Instead of sending more data (high bitrate segments), the system sends lower-bitrate segments and applies SR algorithms at the client device to reconstruct high-quality video, thereby substituting network bandwidth consumption with local computational processing.
Solution Approach 2:
The patent changes the parameter of video representation by transforming low-resolution video segments into high-resolution segments through super-resolution processing. The system selectively applies SR techniques to specific video segments based on their characteristics (e.g., complexity, motion content), thereby improving video quality where needed while maintaining lower bandwidth consumption for segments that don't require enhancement.
2Manufacturing precision
If super-resolution neural networks are applied to enhance video quality on mobile devices, then video quality is improved, but power consumption increases
Solution Approach 1:
The patent applies super-resolution processing selectively to specific regions or segments of video content rather than uniformly to all video data. The system identifies video segments that benefit most from SR enhancement (e.g., static scenes, low-motion segments) and applies processing only to those, thereby reducing overall power consumption while maintaining video quality improvement where it matters most.
Solution Approach 2:
The patent implements a balanced approach where super-resolution is applied partially to video segments based on a cost function that weighs quality improvement against power consumption. Rather than applying SR to all segments (excessive action), the system selectively applies it only when the quality benefit justifies the power cost, achieving optimal trade-off between video quality and energy usage.
3Quantity of substance
If device-adaptive SR-based ABR algorithm is used to reduce bandwidth usage, then bandwidth consumption is reduced, but computational complexity increases
Solution Approach 1:
The patent segments video content into multiple representations with different bitrates and resolutions, then selectively applies super-resolution processing to specific segments based on their characteristics. The ABR algorithm divides video streaming into manageable segments and applies SR selectively, reducing the need to process all video data computationally while maintaining bandwidth efficiency.
Solution Approach 2:
The patent introduces a cost function as an intermediary mechanism that mediates between bandwidth consumption and computational complexity. The cost function evaluates multiple factors including segment quality, bandwidth availability, and device computational capabilities to make intelligent decisions about which segments to enhance with SR, thereby balancing bandwidth usage and computational complexity automatically.
Data Source
AI summary
A mobile device capable of device-adaptive super-resolution based adaptive streaming may include a super-resolution-based adaptive bitrate (SR-based ABR) application configured to receive an input and determine a quality of a representation of a video to request based on the input, the SR-based ABR application being able to compute a cost of requesting the quality of representation based on a cost function. The mobile device may apply an SR network to the representation of the video to upscale the representation of the video to a desired resolution. The mobile device also may include a display configured to play back a segment of the video. A method for SR-based adaptive streaming may include receiving an input, determining whether to request a lower resolution representation of a video segment based on the input, and applying an SR network to the lower resolution representation to upscale it to a desired resolution.