Neural Quantization for Faster Video Encoder RDOQ
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Rate-distortion optimized quantization (RDOQ) in video encoding is resource-intensive and time-consuming due to sequential search methods, making it challenging for real-time video coding on hardware.
Innovation Solution
A neural network-based approach that determines quantization levels for transform coefficients in parallel, using a fully convolutional neural network or autoregressive model to infer optimal quantization levels based on scaled and scalar quantized coefficients, reducing the need for additional hardware circuitry.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional sequential search methods are used for rate-distortion optimized quantization, then quantization optimization is achieved, but processing time and resource consumption increase significantly
Solution Approach 1:
The patent replaces the traditional sequential mechanical search process with a neural network-based parallel processing system. The neural network is trained to predict optimal quantization levels directly, substituting the step-by-step sequential search with a single parallel inference pass that processes all transform coefficients simultaneously, thereby dramatically reducing processing time while maintaining quantization optimization.
Solution Approach 2:
The patent performs preliminary training of the neural network offline using training data that includes transform coefficients and corresponding optimal quantization levels. This preliminary action prepares the model in advance so that during actual video encoding, the pre-trained network can quickly infer quantization levels without performing real-time sequential searches, thus resolving the time-consuming issue.
2Measurement precision
If traditional RDOQ algorithms are used, then rate-distortion optimization is achieved, but hardware resource consumption increases
Solution Approach 1:
The patent substitutes complex hardware-based sequential search circuits with a neural network implementation that can be deployed using standard processors or specialized AI accelerators. The neural network model, being a software-based solution with pre-computed weights, reduces the need for additional dedicated hardware circuitry while maintaining rate-distortion optimization capabilities.
Solution Approach 2:
The patent uses a pre-trained neural network model that encapsulates the rate-distortion optimization knowledge. Instead of implementing complex optimization logic in hardware, the system copies the learned patterns from training data into the network weights, allowing the model to reproduce optimal quantization decisions without requiring the original complex search algorithms to be implemented in hardware.
3Productivity
If parallel processing is implemented using neural networks, then processing speed increases, but model complexity and training requirements increase
Solution Approach 1:
The patent segments the quantization optimization task into two distinct phases: an offline training phase where the neural network learns from training data, and an online inference phase where the pre-trained model processes video data in parallel. This segmentation allows the complex learning process to be separated from the fast processing phase, achieving high processing speed during encoding while managing model complexity through efficient network architecture design.
Data Source
AI summary
A video encoder determines scaled transform coefficients, wherein determining the scaled transform coefficients comprises scaling transform coefficients of a block of the video data according to a given quantization step. The video encoder determines scalar quantized coefficients, wherein determining the scalar quantized coefficients comprises applying scalar quantization to the scaled transform coefficients of the block. Additionally, the video encoder applies a neural network that determines a respective set of probabilities for each respective transform coefficient of the block. The respective set of probabilities for the respective transform coefficient includes a respective probability value for each possible adjustment value in a plurality of possible adjustment values. Inputs to the neural network include the scaled transform coefficients and the scalar quantized coefficients. The video encoder determines, based on the set of probabilities for a particular transform coefficient of the block, a quantization level for the particular transform coefficient.