Optical Flow Prediction Refinement for Video Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding technologies face challenges in efficiently managing bandwidth demand due to the high data requirements of digital video, particularly as the number of connected devices increases, necessitating improved compression methods.
Innovation Solution
The implementation of optical flow-based refinement techniques in video encoding and decoding processes to enhance prediction accuracy and efficiency by modifying prediction samples with gradient components and motion displacements, allowing for more precise conversion between video blocks and bitstream representations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If optical flow based refinement is applied to modify prediction samples with gradient components and motion displacements, then prediction accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent applies preliminary action by performing optical flow based refinement using gradient components and motion displacements before the final prediction coding stage. The gradient components (Gx, Gy) and motion displacements (Vx, Vy) are calculated and applied to modify prediction samples in advance, allowing the decoder to reconstruct higher quality prediction samples without increasing the bitstream complexity. This preliminary refinement resolves the contradiction by improving prediction accuracy while keeping the additional computational requirements confined to the encoding/decoding process rather than the transmission process.
2Productivity
If more precise prediction samples are used in video encoding, then compression efficiency is improved, but processing time increases
Solution Approach 1:
The patent applies parameter changes by modifying the prediction sample values through gradient components and motion displacements derived from optical flow analysis. By changing the parameters of the prediction samples (adding Gx*x + Gy*y offsets based on motion displacements), the method achieves better prediction accuracy and compression efficiency. The mathematical operations involved are relatively simple gradient calculations and linear combinations, which can be efficiently implemented, thus improving compression efficiency without excessive processing time penalty.
Data Source
AI summary
A method of video processing includes determining a refined prediction sample P′(x,y) at a position (x,y) in a video block by modifying a prediction sample P(x,y) at the position (x,y) with a first gradient component Gx(x, y) in a first direction estimated at the position (x,y) and a second gradient component Gy(x, y) in a second direction estimated at the position (x,y) and a first motion displacement Vx(x,y) estimated for the position (x,y) and a second motion displacement Vy(x,y) estimated for the position (x,y), where x and y are integer numbers, and performing a conversion between the video block and a bitstream representation of the video block using a reconstructed sample value Rec(x,y) at the position (x,y) that is obtained based on the refined prediction sample P′(x,y) and a residue sample value Res(x,y).