Video Interpolation with Directional Energy Occlusion Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video interpolation methods face challenges in handling occlusions and disocclusions, leading to artifacts like ghosting or halo, and require significant computational resources and buffer size, making them inefficient.
Innovation Solution
A video interpolation method that determines interpolation parameters by associating directions of regularity between consecutive frames, allowing for efficient interpolation using only two consecutive frames, reducing computational complexity and buffer requirements by comparing directional energies and directions of regularity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If preliminary detection of occlusion/disocclusion areas is performed using multiple consecutive frames, then occlusion handling accuracy is improved, but line buffer size requirement increases and computational cost increases
Solution Approach 1:
The patent extracts only the necessary information (direction of regularity and minimized directional energy) from each frame pair, storing only these parameters in the line buffer rather than complete frame data. This extraction principle reduces buffer size while maintaining sufficient information for occlusion detection and interpolation decisions.
Solution Approach 2:
The patent performs preliminary calculation of direction of regularity and minimized directional energy for pixels in reference frames before interpolation is needed. These pre-computed parameters are stored in the buffer and reused during interpolation, avoiding redundant computations and reducing real-time processing load.
2Measurement precision
If preliminary detection of occlusion/disocclusion areas is performed using multiple consecutive frames, then occlusion handling accuracy is improved, but computational cost increases
Solution Approach 1:
The patent extracts only the essential parameters (direction of regularity and minimized directional energy) needed for occlusion detection, avoiding computation of complete motion fields or other redundant information. This selective extraction significantly reduces computational cost while maintaining detection accuracy.
Solution Approach 2:
The patent performs preliminary computation of direction of regularity and minimized directional energy values before the interpolation step. These pre-computed values are stored and reused, eliminating the need for repeated complex computations during real-time interpolation and occlusion handling.
3Productivity
If directional interpolation is applied in occlusion regions, then interpolation speed is maintained, but artifacts such as ghosting or halo are inevitable
Solution Approach 1:
The patent uses the minimized directional energy as a feedback criterion to evaluate the quality of directional interpolation. By comparing directional energies from different reference frames, the algorithm detects occlusion regions where directional interpolation would produce artifacts and switches to alternative strategies, thus eliminating ghosting and halo while maintaining efficiency.
Solution Approach 2:
The patent changes the interpolation parameter selection based on the minimized directional energy criterion. In non-occlusion regions, standard directional interpolation parameters are used for speed. In occlusion regions detected through energy comparison, the parameters are changed to select pixels from the appropriate reference frame, eliminating artifacts.
4Object-generated harmful factors
If one-sided directional interpolation is used to reduce artifacts, then artifact reduction is achieved, but it is necessary to determine the proper motion knowing that only one of the two reference frames provides the right pixel information
Solution Approach 1:
The patent uses minimized directional energy as a feedback mechanism to automatically determine which reference frame provides the correct pixel information. By comparing the directional energies computed from both reference frames, the algorithm identifies the frame with the lower energy (better match) and selects it for one-sided interpolation, simplifying motion determination without reducing artifact reduction effectiveness.
Data Source
AI summary
A direction of regularity, which minimizes a directional energy computed from pixel values of consecutive first and second frames of an input video sequence, is respectively associated with each pixel of the first frame and with each pixel of the second frame. Another direction of regularity (vz), which minimizes a directional energy computed from pixel values of the first and second frames, is also associated with an output pixel (z) of a frame of an output video sequence, located in time between the first and second frames. For processing such output pixel, the respective minimized directional energies for the output pixel, at least one pixel (z′) of the first frame and at least one pixel (z″) of the second frame are compared to control an interpolation performed to determine a value of the output pixel. The interpolation uses pixel values from at least one of the first and second frames of the input video sequence depending on the comparison of the minimized directional energies.