Optical Flow Video Coding Refinement via Gradient Clipping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding methods face challenges in efficiently compressing high-resolution videos due to increased bandwidth demands and the need for improved coding efficiency, particularly in existing standards like HEVC and future standards like VVC, where existing filtering techniques may not adequately address artifacts and residual errors.
Innovation Solution
The proposed solution involves refining video processing methods by applying clipping operations and filtering processes based on weighted averages and gradient values, specifically using a geometry transformation-based adaptive loop filter (GALF) and optical flow techniques to enhance prediction refinement and reduce residual errors, thereby improving coding efficiency and video quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If existing filtering techniques are used in video coding, then device complexity is reduced, but manufacturing precision (video quality and artifact reduction) deteriorates
Solution Approach 1:
The patent divides the filtering process into multiple stages: first applying a deblocking filter to reduce block artifacts, then applying an adaptive loop filter to further refine video quality. This segmentation allows each filter to specialize in specific types of artifacts, improving overall video quality without requiring a single overly complex filtering system.
Solution Approach 2:
The adaptive loop filter uses gradient calculations to identify regions with different characteristics (e.g., edges vs. smooth areas) and applies different filtering strengths to different regions. This local adaptation improves video quality in complex areas while maintaining efficiency in simpler regions, resolving the contradiction between quality and complexity.
2Measurement precision
If gradient calculation is applied in optical flow coding, then measurement precision (motion estimation accuracy) is improved, but loss of time (computational time) increases
Solution Approach 1:
The patent calculates gradient values for all pixels before performing optical flow estimation. By pre-computing these gradients, the actual motion estimation process can proceed more efficiently using the prepared gradient data, reducing the computational time during the critical encoding/decoding phase while maintaining high motion estimation accuracy.
Solution Approach 2:
The patent applies gradient calculation to all pixels in a block, which may seem excessive, but this comprehensive approach enables more accurate optical flow estimation. The additional computational effort in gradient calculation is offset by the efficiency gains in subsequent motion compensation steps, where the pre-computed gradients are reused.
Data Source
AI summary
Devices, systems and methods for sample refinement and filtering method for video coding are described. In an exemplary aspect, a method for video processing includes modifying for a conversion between a block of a video and a bitstream representation of the video, a refinement value for a prediction sample in the block by applying a clipping operation to refinement value. The refinement value is derived based on a gradient value of an optical flow coding process. An output of the clipping operation is within a range. The method also includes refining the prediction sample based on the refinement value and performing the conversion based on the refined prediction sample.