Neural Motion Compensation for Sub-Pixel Video Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing fractional pixel motion compensation technologies struggle to adapt to diverse video contents and complex encoding environments, leading to low encoding and decoding efficiency due to non-stationary natural picture signals and nonlinearity of encoding noise.
Innovation Solution
Implement motion compensation enhancement using a neural network model, such as a Convolutional Neural Network (CNN), to perform super-resolution and quality enhancement on matching blocks, followed by down-sampling to generate high-precision sub-pixel samples, improving prediction precision and reducing computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing fractional pixel motion compensation technical schemes are used, then motion compensation can be performed, but encoding and decoding efficiency is low due to non-stationary natural picture signals and nonlinearity of encoding noise
Solution Approach 1:
The patent applies dynamics by transitioning from fixed interpolation filters to adaptive neural network models that dynamically adjust to different video contents and encoding environments. The neural network is trained on diverse video data to adapt its parameters, enabling it to handle non-stationary signals and nonlinear encoding noise effectively, thus improving both efficiency and adaptability
Solution Approach 2:
The patent changes parameters by using learned interpolation filters from neural networks instead of traditional fixed filters. The neural network parameters are optimized during training to minimize reconstruction error, allowing the system to adapt to different video characteristics and encoding conditions, thereby resolving the contradiction between efficiency and adaptability
2Measurement precision
If motion compensation enhancement is performed using neural network models, then prediction precision is improved, but computational complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-training the neural network model offline to learn optimal interpolation filters. Once trained, the model parameters are fixed and can be deployed with minimal real-time computation. This pre-computation approach achieves high prediction precision while reducing online computational complexity, as the heavy lifting is done during the offline training phase
Solution Approach 2:
The patent substitutes traditional mechanical interpolation filter systems with neural network-based learning systems. The neural network replaces fixed mathematical interpolation formulas with learned patterns from data, achieving superior precision while the pre-trained model reduces real-time computational burden through efficient inference
3Device complexity
If traditional interpolation filters are used for fractional pixel motion compensation, then computational complexity is low, but adaptability to diverse video contents and complex encoding environments is poor
Solution Approach 1:
The patent applies self-service by enabling the neural network to automatically adapt to different video contents and encoding environments through its learned parameters. The model learns from diverse training data and can self-adjust its behavior without manual intervention, maintaining low computational complexity during inference while achieving high adaptability across different scenarios
Data Source
AI summary
An encoding method includes: a first matching block of a current block is determined; motion compensation enhancement is performed on the first matching block to obtain at least one second matching block; motion information of the current block is determined according to the at least one second matching block; and the current block is encoded according to the motion information.