Per-Title Bitrate Ladders With DNN Enhancement for Mixed CPU-GPU Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video streaming technologies optimize bitrate ladders based on CPU or GPU processors, compromising video quality for the other type of processor, leading to suboptimal performance for both CPU-only and GPU-available devices.
Innovation Solution
A scalable per-title encoding (SPTE) approach that constructs a bitrate ladder as a base layer for CPU-only devices and adds an enhancement layer with content-aware video super-resolution deep neural networks (DNNs) for GPU-available devices, using DNN compression techniques to reduce bitrate overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a single bitrate ladder is used for all video contents, then the system complexity is reduced and ease of operation is improved, but video quality suffers for both CPU-only and GPU-available devices
Solution Approach 1:
The patent segments video content into scene segments based on visual similarity, allowing different bitrate ladder optimizations for different segments. This enables quality improvement without requiring a completely separate bitrate ladder for every video, thus balancing complexity and quality.
Solution Approach 2:
The patent applies local quality optimization by training content-aware super-resolution models specifically for difficult-to-encode scene segments rather than uniformly across all content. This focuses computational resources where they provide the most quality improvement while maintaining acceptable performance for easier segments.
2Manufacturing precision
If bitrate ladder is optimized for CPU-only devices, then video quality is improved for CPU-only devices, but video quality deteriorates for GPU-available devices
Solution Approach 1:
The patent implements dynamic bitrate ladder optimization by detecting device type (CPU-only vs. GPU-available) and adapting the bitrate ladder selection accordingly. GPU-available devices receive bitrate ladders optimized for higher quality with super-resolution enhancement, while CPU-only devices receive appropriately optimized ladders without requiring real-time super-resolution processing.
Solution Approach 2:
The patent changes the parameter set used for bitrate ladder construction based on device capabilities. For GPU-available devices, the system uses parameters that leverage super-resolution models, while for CPU-only devices, it uses parameters optimized for standard decoding performance, thus adapting to different device characteristics.
3Manufacturing precision
If per-title encoding is performed to optimize bitrate ladder, then video quality is improved, but encoding time and computational cost increase
Solution Approach 1:
The patent applies partial optimization by focusing super-resolution model training only on difficult-to-encode scene segments rather than processing entire videos uniformly. This selective approach achieves quality improvement where most needed while significantly reducing encoding time compared to processing all content at maximum quality.
Solution Approach 2:
The patent performs preliminary scene segment detection and classification before bitrate ladder optimization, identifying which segments require super-resolution enhancement. This preliminary action allows the system to prepare optimized bitrate ladders in advance for only the segments that need them, reducing overall encoding time.
4Manufacturing precision
If content-aware video super-resolution is applied, then video quality is improved, but bitrate overhead increases
Solution Approach 1:
The patent introduces a lightweight super-resolution model as an intermediary between the base video encoding and the final rendered output. This intermediary processes only the necessary enhancement operations on difficult segments, achieving quality improvement while maintaining controlled bitrate overhead through selective application rather than universal processing.
Data Source
AI summary
A scalable per-title encoding technique may include detecting scene cuts in an input video received by an encoding network or system, generating segments of the input video, performing per-title encoding of a segment of the input video, training a deep neural network (DNN) for each representation of the segment, thereby generating a trained DNN, compressing the trained DNN, thereby generating a compressed trained DNN, and generating an enhanced bitrate ladder including metadata comprising the compressed trained DNN. In some embodiments, the method also may include generating a base layer bitrate ladder for CPU devices, and providing the enhanced bitrate ladder for GPU-available devices.