Interpolated Frame Generation via DFD-Based Mixing Coefficients
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Standard motion compensated frame interpolation techniques suffer from noticeable artefacts in zones with temporal discontinuity, such as covered or uncovered areas, due to failures in the motion estimator, leading to incorrect motion vectors and resulting artefacts like echoes.
Innovation Solution
A device that generates an interpolated frame by calculating mixing coefficients based on displaced frame difference (DFD) and frame difference (FD) values, blending motion compensated and linearly filtered frames to reduce artefacts, with coefficients determined by the ratio of DFD to the sum of DFD and FD, and applying a regularisation function to prioritize the motion compensated frame when DFD and FD are low.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If motion compensated frame interpolation is used to improve image quality and temporal continuity, then temporal continuity is improved, but artefacts such as echoes appear in zones with temporal discontinuity due to motion estimator failures
Solution Approach 1:
The patent changes the parameter used for mixing coefficients from motion vector amplitude to frame difference values (DFD and FD). By using frame difference values that directly measure pixel intensity differences, the system can identify zones with temporal discontinuity more accurately and adjust mixing coefficients accordingly, reducing artefacts while maintaining temporal continuity in stable regions
Solution Approach 2:
The patent introduces frame difference values (DFD and FD) as intermediary parameters to mediate between motion compensated interpolation and linear filtering. These frame difference values serve as a reliable indicator to determine when to trust motion vectors and when to rely more on linear filtering, thus reducing artefacts in discontinuous zones
2Object-generated harmful factors
If linear filtering is used to reduce artefacts in some zones, then artefact reduction is improved, but temporal continuity and image quality deteriorate in zones with significant motion
Solution Approach 1:
The patent dynamically adjusts the mixing coefficient for each pixel based on locally calculated frame difference values. This allows the system to adaptively choose between motion compensated interpolation and linear filtering on a per-pixel basis, maintaining temporal continuity in regions with low frame differences while reducing artefacts in regions with temporal discontinuity
Solution Approach 2:
The patent applies different interpolation strategies to different spatial zones based on local frame difference characteristics. By calculating DFD and FD values for each pixel and using them to determine local mixing coefficients, the system optimizes image quality locally rather than applying a global mixing strategy
3Reliability
If mixing coefficients are determined by motion vector amplitude, then motion compensation is improved, but reliability deteriorates when motion vectors are defective in covered or uncovered zones
Solution Approach 1:
The patent uses frame difference values (DFD and FD) as feedback to evaluate the reliability of motion vectors. By comparing the frame difference before and after motion compensation, the system can detect when motion vectors are defective and adjust mixing coefficients accordingly, improving reliability in zones where motion estimation fails
Data Source
AI summary
The present invention is situated within the frame or video processing domain and more particularly in the domain of video interpolation. It relates to a device for generating an interpolated frame from source frames temporarily surrounding it. This device comprises motion estimation means, motion compensated interpolation means, linear interpolation means, means for mixing the frame produced by the motion compensated interpolation means with the frame produced by the linear interpolation means. According to the invention, the mixing coefficients assigned to these two frames depend on the displaced frame difference and/or the frame difference between the source frames surrounding this interpolated frame and/or estimated motion vectors.