Receptive-field CNNs for Video Coding Mode Decision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern video codecs face increased computational complexity due to improved coding efficiency, requiring more time to determine the best mode decision for encoding, which can lead to suboptimal results and increased resource consumption.
Innovation Solution
A convolutional neural network (CNN) is employed for determining mode decisions in video coding, utilizing feature extraction layers and classifiers to infer partition decisions for sub-blocks, reducing computational complexity by applying non-overlapping convolution operations and kernel size adjustments, thereby optimizing encoding processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If traditional video encoding methods are used to improve coding efficiency, then compression performance is improved, but computational complexity increases
Solution Approach 1:
The patent segments the video block into multiple sub-blocks and processes each sub-block independently through separate classifier networks. This segmentation allows parallel processing of different regions, reducing the overall computational complexity while maintaining coding efficiency through localized mode decisions.
Solution Approach 2:
The patent dynamically adjusts the kernel size of convolution operations based on the block size and partitioning level. By adapting the kernel size (e.g., using 3x3 for 16x16 blocks, 5x5 for 32x32 blocks), the system optimizes computational complexity for different scales while preserving the necessary feature extraction capability for accurate mode decision.
2Manufacturing precision
If more computation time is allocated to mode decision, then coding efficiency is improved, but processing speed decreases
Solution Approach 1:
By dividing the block into sub-blocks and processing them in parallel through multiple classifiers, the patent reduces the sequential computation time required for mode decision. Each classifier operates independently on its sub-block, enabling simultaneous processing that accelerates overall encoding speed.
Solution Approach 2:
The patent replaces traditional exhaustive search or complex optimization algorithms with a trained neural network classifier system. This substitution leverages pre-computed knowledge from training data, allowing fast inference during encoding without requiring intensive real-time computation, thus improving processing speed while maintaining coding efficiency.
3Measurement precision
If conventional classification layers are used, then feature classification is achieved, but computational complexity increases due to full feature dimension processing
Solution Approach 1:
The patent extracts and processes only the most relevant features for mode decision by using 1x1 convolution kernels that selectively filter and reduce the feature dimension. This extraction approach removes redundant features while retaining critical information, reducing computational complexity without significantly compromising classification accuracy.
Solution Approach 2:
Different regions of the block are processed with appropriate kernel sizes and feature dimensions tailored to their specific characteristics. The patent applies local quality by adapting the classification approach to each sub-block's needs, using smaller kernels for regions requiring less detail and larger kernels for regions needing more comprehensive analysis, thereby optimizing the balance between accuracy and complexity.
Data Source
AI summary
Convolutional neural networks (CNN) that determine a mode decision (e.g., block partitioning) for encoding a block include feature extraction layers and multiple classifiers. A non-overlapping convolution operation is performed at a feature extraction layer by setting a stride value equal to a kernel size. The block has a N×N size, and a smallest partition output for the block has a S×S size. Classification layers of each classifier receive feature maps having a feature dimension. An initial classification layer receives the feature maps as an output of a final feature extraction layer. Each classifier infers partition decisions for sub-blocks of size (αS)×(αS) of the block, wherein α is a power of 2 and α=2, . . . , N/S, by applying, at some successive classification layers, a 1×1 kernel to reduce respective feature dimensions; and outputting by a last layer of the classification layers an output corresponding to a N/(αS)×N/(αS)×1 output map.