Frame Interpolation Using Denoised Component Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing systems struggle to accurately generate interpolation frames when frame images include noise, leading to reduced accuracy in motion vector detection and interpolation image generation.
Innovation Solution
An image processing apparatus and method that reduces noise components in frame images by generating first and second primary component images, estimates displacement based on these components, and generates interpolation images corresponding to different acquisition times, using techniques such as frequency component processing, filtering, and learned models to enhance accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If motion vectors are calculated based on frame images including noises, then the system can process real-world images, but the motion vectors cannot be accurately detected
Solution Approach 1:
The patent segments the frame image into multiple components: a first component (e.g., background or static elements) and a second component (e.g., foreground or moving elements). By separating the noisy image into these components, the system can process each component independently, reducing the impact of noise on motion vector detection accuracy.
Solution Approach 2:
The patent introduces an intermediary processing step that generates a denoised version of the frame image before motion vector calculation. This intermediary processed image serves as a mediator between the noisy input image and the motion vector detection algorithm, enabling accurate detection while maintaining adaptability to real-world noisy conditions.
2Productivity
If interpolation frames are generated from noisy frame images, then the frame rate can be increased, but the interpolation images cannot be accurately generated
Solution Approach 1:
The patent segments the interpolation generation process into multiple stages: first separating the frame image into components, then generating motion vectors from the denoised first component, and finally synthesizing the interpolation image by combining processed components. This segmentation enables high frame rate generation while maintaining accuracy through systematic processing of each component.
Solution Approach 2:
The patent performs preliminary denoising and component separation before generating motion vectors and interpolation images. By preparing the frame images in advance through noise reduction and component decomposition, the system ensures that subsequent interpolation operations work with clean data, achieving both high frame rate and high accuracy.
3Measurement precision
If noise reduction is applied to frame images, then the motion vector detection accuracy improves, but the processing time increases
Solution Approach 1:
The patent segments the image processing into component separation and noise reduction steps that can be performed in parallel or with optimized sequencing. By dividing the complex processing into manageable segments (component extraction, denoising, motion vector calculation), the system reduces overall processing time while maintaining the benefits of noise reduction for accurate motion vector detection.
Data Source
AI summary
An image processing apparatus according to this invention includes an image acquirer configured to acquire the set of frame images, and an image processor. The image processor is configured to generate, based on first and second frame images, first and second primary component images such that the primary component images each include first component having a reduced noise component, respectively, to generate a second component image including a second component that is different from the first component based on one of the two frame images, and to generate an interpolation image that corresponds to an acquisition time different from acquisition times of the two frame images based on a displacement estimated based on the first component of the two first component images and the second component image.