Video Frame Interpolation Using Trust-Based Pixel Smoothing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Frame rate conversion in video sequences often introduces halo artefacts due to inaccurate motion vector determination in occluded and revealed areas, leading to perceptual distortion and visible artefacts like 'halo' that can draw attention and degrade video quality.
Innovation Solution
A method and system for generating output frames in video sequences that involve determining trust indications for pixel values in motion-compensated images, using these indications to selectively combine motion-compensated and smoothed images, thereby reducing halo artefacts by smoothing areas with low trust and using original values for trusted pixels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If motion vectors are used to generate interpolated frames, then frame rate conversion is achieved, but halo artefacts and perceptual distortion occur
Solution Approach 1:
The patent divides the image processing into separate stages: motion compensation generates a first motion-compensated image, then a smoothing operation produces a second motion-compensated image. These segmented processing stages allow selective combination of results to reduce halo artefacts while maintaining frame rate conversion capability.
Solution Approach 2:
The patent applies different processing qualities to different regions by using trust indications to identify pixels prone to halo artefacts. Only specific pixels (those with low trust indications) are smoothed, while other pixels retain their original motion-compensated values, achieving local quality optimization.
2Object-affected harmful factors
If smoothing is applied to reduce halo artefacts, then perceptual distortion is reduced, but image detail and sharpness are lost
Solution Approach 1:
The patent applies smoothing selectively only to pixels identified as prone to halo artefacts through trust indications, rather than smoothing the entire image. This local application preserves image detail in regions not affected by halos while reducing perceptual distortion only where needed.
Solution Approach 2:
The patent applies partial smoothing action rather than complete smoothing to the entire image. By using trust indications to identify only the necessary pixels for smoothing, the patent achieves the minimum required smoothing action to reduce halo artefacts without excessive smoothing that would degrade overall image quality.
3Object-affected harmful factors
If trust indications are used to selectively combine images, then halo artefacts are reduced, but processing complexity increases
Solution Approach 1:
The trust indications are determined automatically based on the motion compensation process itself, using the existing motion data and pixel information. The system self-evaluates which pixels are prone to halo artefacts without requiring external intervention or complex additional algorithms, reducing processing complexity.
Data Source
AI summary
In frame rate conversion, output frames (e.g. interpolated frames) are generated for inclusion in a video sequence. A first motion-compensated image may be generated for representing an output frame, e.g. by using motion estimation based on one or more of the existing frames of the video sequence. At least part of the first motion-compensated image is smoothed to determine a smoothed motion-compensated image. Trust indications can be determined for the pixels of the first motion-compensated image to indicate levels of trust in the pixel values. The trust indications may be used to determine how to selectively combine the pixels of the first motion-compensated image and the pixels of the smoothed motion-compensated image to thereby generate the output frame.