Neural Video Coding Syntax for Standard Decoder Compatibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies, such as HEVC, face inefficiencies in determining optimal partitioning and quantization parameters due to brute-force searches, affecting compression efficiency and compatibility with standard decoders.
Innovation Solution
A neural network-based approach that generates syntax elements processable by standard codecs, utilizing multiple neural networks for intra- and inter-prediction, motion estimation, and adaptive normalization to optimize coding unit partitions and quantization, enabling efficient entropy encoding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If brute-force search is used to determine optimal partitioning and quantization parameters in HEVC, then encoding precision is improved, but computational complexity and encoding time increase significantly
Solution Approach 1:
The patent replaces the mechanical brute-force search system with a neural network-based system. The neural network is trained offline to learn optimal partitioning and quantization parameter selections, then during encoding it directly predicts these parameters without exhaustive search, substituting computational mechanics with learned intelligence.
Solution Approach 2:
The neural network is trained in advance using offline learning with ground truth data from brute-force search results. This preliminary training phase stores optimal parameter selections, allowing the encoder to skip actual brute-force searches during operation while maintaining encoding precision.
2Productivity
If neural network-based video coding is implemented, then compression efficiency is improved, but compatibility with standard decoders deteriorates
Solution Approach 1:
The patent introduces syntax elements as an intermediary between the neural network and standard codecs. The neural network generates syntax elements that conform to existing codec standards, allowing standard decoders to interpret and process the encoded data without modification while still benefiting from neural network-based compression.
Solution Approach 2:
The patent designs the neural network to output multiple types of syntax elements (intra-prediction, inter-prediction, partitioning) that are universally compatible with standard codec frameworks. This multi-functionality allows a single neural network-based encoder to work with various standard decoders maintaining broad compatibility.
3Measurement precision
If multiple neural networks are used for intra- and inter-prediction, then coding performance is improved, but device complexity increases
Solution Approach 1:
The patent segments the video coding task into separate neural networks for different prediction types (intra-prediction, inter-prediction). Each neural network is specialized for its specific function, improving performance while allowing modular implementation and independent optimization of each component.
Data Source
AI summary
An apparatus with video processing includes: one or more processors configured to: generate a syntax element processable by a target standard codec by inputting a quantization parameter, a pre-decoded reference image, and a plurality of frames comprised in a video to a neural network and compressing the plurality of frames, and generate a bitstream by performing entropy encoding on the syntax element.