Motion Vector Field Compression with Neural Networks for Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for compressing and reconstructing motion vector fields in video coding are inefficient, leading to suboptimal video coding efficiency and encoding efficiency.
Innovation Solution
A neural network-based approach is employed to generate, compress, and reconstruct motion vector fields using multiple neural network layers, including convolutional layers, for spatial sampling and normalization based on picture order count differences, with quantization and storage for subsequent motion prediction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional methods are used to compress and reconstruct motion vector fields, then the processing is simple, but the video coding efficiency and encoding efficiency are suboptimal
Solution Approach 1:
The patent replaces conventional mechanical compression methods with a neural network-based system. The neural network learns optimal compression and reconstruction operations through training, substituting traditional algorithmic approaches with data-driven models that achieve superior coding efficiency while managing complexity through automated learning processes.
Solution Approach 2:
The patent changes the parameters and operations used in motion vector field compression by training neural networks to perform non-standard compression operations. The networks learn optimal parameter transformations during training, enabling efficient compression that goes beyond conventional linear operations while maintaining manageable system complexity through learned parameter relationships.
2Productivity
If motion vector field compression is performed without spatial sampling, then the data is preserved completely, but the encoding efficiency is reduced
Solution Approach 1:
The patent applies partial spatial sampling to the motion vector field, processing only the most informative regions rather than the entire field. The neural network learns to identify and process critical spatial locations, achieving efficient encoding by applying compression operations selectively to portions of the motion vector field that contain the most essential motion information.
Solution Approach 2:
The patent replaces conventional mechanical sampling methods with neural network-based spatial sampling. The networks learn optimal sampling patterns through training, automatically identifying which spatial regions require processing and how to sample them, thereby achieving high encoding efficiency while minimizing information loss through intelligent, data-driven selection of critical motion regions.
3Measurement precision
If motion vectors are not normalized by POC difference, then the processing is simpler, but the motion prediction accuracy is reduced
Solution Approach 1:
The patent performs normalization of motion vectors by picture order count (POC) difference as a preliminary action before compression. The neural network is trained to expect normalized input, so the normalization operation is performed in advance to prepare the data for optimal compression. This preliminary normalization ensures that the subsequent compression operations work with standardized data, improving prediction accuracy while the complexity is managed through the learned expectations of the trained network.
Data Source
AI summary
A neural network-based image processing method and apparatus, according to an embodiment of the present invention, may generate a motion vector field by using motion information used in motion prediction in processing units, included in the present picture, and generate a tensor of the motion vector field by performing compression on the motion vector field on the basis of a neural network including a plurality of neural network layers.