Video Encoder Super Resolution Motion Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding standards, such as H.264/AVC, face limitations in accurately capturing real motion due to the use of rectangular image blocks and insufficient sub-pixel accuracy, leading to incomplete regeneration of high spatial frequencies and increased data rates.
Innovation Solution
The implementation of a super resolution algorithm to calculate high-resolution reference image blocks for local motion compensation, enabling more precise temporal prediction and improved motion estimation, which is then used to enhance the encoding and decoding process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sub-pixel motion compensation with interpolation is used to improve motion accuracy, then prediction accuracy is improved, but high spatial frequencies above Nyquist frequency cannot be regenerated
Solution Approach 1:
The patent transitions from spatial domain interpolation to frequency domain processing by applying Discrete Cosine Transform (DCT) to generate super-resolution images. This dimensional change allows regeneration of high spatial frequencies that were lost in the original image by operating in the frequency domain where these components can be reconstructed and then transformed back to spatial domain.
Solution Approach 2:
The patent changes the resolution parameter by generating super-resolution images with higher spatial frequencies than the original image. By modifying the frequency content through DCT and inverse DCT operations, the system recovers and enhances high spatial frequency components that were not present in the original sub-sampled image data.
2Device complexity
If rectangular image blocks are used for motion estimation, then computational complexity is reduced, but accuracy of capturing real motion is insufficient
Solution Approach 1:
The patent combines multiple processing techniques into a composite approach: standard rectangular block motion estimation is combined with super-resolution image generation using DCT. This composite method maintains the simplicity of rectangular block matching while adding a preprocessing step that enhances motion estimation accuracy by providing higher quality reference images with recovered high spatial frequencies.
3Quantity of substance
If standard motion compensation is used to maintain data rate, then bandwidth efficiency is maintained, but prediction error is increased leading to lower signal quality
Solution Approach 1:
The patent performs preliminary action by generating super-resolution reference images before the motion estimation and compensation process. By pre-processing the reference images to recover high spatial frequencies and improve quality, the subsequent motion compensation operates on enhanced data, resulting in lower prediction errors and better signal quality without increasing the final encoded data rate.
Data Source
AI summary
An encoder encodes a video signal formed of video frames, each including image blocks. The encoder includes a processing unit which calculates at least one high resolution reference image block on the basis of previously encoded image blocks by executing a super resolution algorithm to perform a local motion compensation; and a motion compensation unit which calculates on the basis of the calculated high resolution reference image block a temporal predictor which is subtracted from a current image block of the video signal. Together, the encoder and a corresponding decoder improve the signal quality of a video signal significantly.