Artefact Detection in Video Frame Interpolation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Frame rate conversion in video sequences often results in artefacts, particularly the erroneous elimination of subtle image features, leading to flickering issues when displayed to users, due to the inability of motion estimators to accurately track small or delicate features against dominant backgrounds.
Innovation Solution
An artefact detector system that processes pixel values in input frames to identify blocks representing image features, matches these blocks between frames, and characterizes corresponding blocks in interpolated frames as artefacts, enabling correction to restore the missing features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If motion estimation is performed using block-based methods to generate interpolated frames, then frame rate conversion is achieved, but subtle image features are erroneously eliminated causing flickering artefacts
Solution Approach 1:
The image is divided into multiple blocks for motion estimation processing. Each block is independently analyzed to determine motion vectors, allowing fine-grained control over different regions of the image. This segmentation enables the system to process subtle features in specific blocks without being overwhelmed by the entire frame, thereby maintaining feature accuracy while achieving frame rate conversion.
Solution Approach 2:
The system applies different processing strategies to different blocks based on their local characteristics. Blocks containing subtle image features are identified and processed with higher precision motion estimation techniques, while other blocks use standard methods. This local quality approach ensures that subtle features are preserved in critical areas without sacrificing overall conversion efficiency.
2Ease of operation
If block-based motion estimation is used to track image motion, then frame interpolation is enabled, but subtle features against dominant backgrounds cannot be accurately tracked
Solution Approach 1:
Before performing standard motion estimation, the system pre-processes blocks to identify and flag those containing subtle image features. This preliminary action involves analyzing block characteristics such as contrast, edge density, and feature prominence. By identifying subtle feature blocks in advance, the system can apply enhanced tracking methods specifically to these regions, ensuring accurate feature tracking while maintaining overall interpolation efficiency.
Solution Approach 2:
The system incorporates feedback mechanisms where motion estimation results are continuously evaluated against expected feature persistence. When subtle features are detected in output frames or when motion vectors indicate potential feature loss, the system adjusts subsequent motion estimation parameters for affected blocks. This feedback loop ensures that subtle features maintain their visibility and correctness throughout the interpolation process.
Data Source
AI summary
An artifact detector detects artifacts in a video sequence comprising interpolated frames generated by performing motion estimation. The detector comprises a pixel processor which processes pixel values in first and second input frames of the video sequence to identify respective blocks of pixels representing an image feature. A feature-matching module matches identified blocks in the first input frame with corresponding identified in the second input frame to determine matched block pairs. For each matched block pair, a feature-detector module: (i) identifies a block of an interpolated frame between the first and second input frames corresponding to the matched block pair, and (ii) characterizes the corresponding block of the interpolated frame as an artifact in dependence on a determination of whether said corresponding block does or does not represent the image feature. A corrector module corrects the interpolated frame based on the characterization of the blocks of the interpolated frame.