Video Encoding Mode Selection Using Hierarchical Machine Learning Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video encoding methods spend a significant portion of computation time on selecting the optimal encoding mode, which can be inefficient, especially in real-time live video encoding applications, due to the reliance on iterative trial-and-error processes.
Innovation Solution
The use of machine learning models to predictively select encoding modes for video encoders, such as those following the Advanced Video Coding (AVC) standard, by employing a hierarchy of machine learning models to determine optimal encoding modes for each macroblock of a frame, reducing computation costs and improving efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If iterative trial-and-error processes are used to select encoding modes, then encoding quality can be optimized, but computation time increases significantly
Solution Approach 1:
The patent applies preliminary action by pre-training machine learning models to predict optimal encoding modes before actual encoding occurs. The models are trained offline using historical encoding data and patterns, allowing them to make rapid predictions during real-time encoding without requiring iterative trial-and-error processes. This shifts the computationally intensive quality optimization work to an offline phase.
Solution Approach 2:
The patent replaces the mechanical iterative trial-and-error encoding process with a machine learning-based predictive system. Instead of physically testing multiple encoding modes and selecting the best one through computation, the system uses trained ML models to directly predict optimal modes, substituting iterative mechanical computation with intelligent prediction.
2Loss of time
If machine learning models are used to predictively select encoding modes, then computation time is reduced, but model complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the encoding mode selection task into multiple independent macroblocks within video frames. Each macroblock is processed separately by the machine learning model, allowing parallel computation and reducing the overall complexity burden on any single model. This segmentation enables the system to handle complex video data through simpler, independent processing units.
Solution Approach 2:
The patent utilizes parameter changes by adjusting model complexity based on specific encoding conditions and macroblock characteristics. The system can switch between different prediction models or adjust model parameters dynamically depending on the video content, scene complexity, and encoding requirements, thereby managing computational complexity while maintaining prediction accuracy.
Data Source
AI summary
Techniques for training and using machine learning models for video encoding mode selection are described. According to some embodiments, a computer-implemented method includes receiving a live video at a content delivery service, extracting one or more features for a plurality of macroblocks of a frame of the live video, determining an encoding mode from a plurality of encoding modes for each of the plurality of macroblocks of the frame with a machine learning model based at least in part on an input of the one or more features, performing a real time encode of the frame of the live video based at least in part on the determined encoding modes to generate an encoded frame by the content delivery service, and transmitting the encoded frame from the content delivery service to a viewer device.