Event-Guided Motion Compensation for Blur-Free Video Denoising
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing noise reduction methods for moving images, such as averaging multiple frames, cause blurring in moving subjects and do not effectively utilize high-speed event-based vision sensors for accurate motion estimation and noise reduction.
Innovation Solution
A moving image noise reduction apparatus and method that decomposes image data into low-frequency and high-frequency components, uses an event-based sensor to estimate motion vectors at a higher frame rate, and performs motion compensation to reduce noise by combining these components.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple video frames are averaged to reduce noise, then noise reduction effectiveness is improved, but moving subjects become blurred
Solution Approach 1:
The image data is decomposed into multiple frequency components (low-frequency component L, and high-frequency components H1, H2, H3). This segmentation allows different processing strategies to be applied to different frequency bands, enabling noise reduction in the low-frequency domain while preserving motion details in the high-frequency domain, thus resolving the contradiction between noise reduction and subject clarity.
Solution Approach 2:
Different quality treatments are applied to different frequency components. The low-frequency component undergoes averaging for noise reduction, while the high-frequency components are processed separately to preserve motion information. This local quality differentiation allows the system to achieve both noise reduction and motion clarity simultaneously.
2Measurement precision
If event sensor data is read and processed at high frame rate, then motion estimation accuracy is improved, but data processing speed requirement increases
Solution Approach 1:
The high-frequency component H2 from the event sensor is further decomposed into multiple sub-components (H2-1, H2-2, H2-3). This segmentation reduces the data volume that needs to be processed at the original high frame rate, making it feasible to perform accurate motion estimation without overwhelming the processing system.
Solution Approach 2:
The system processes only the necessary portions of the event sensor data at full resolution and frame rate, while using the decomposed high-frequency components for supplementary motion information. This partial processing approach maintains motion estimation accuracy while reducing the overall processing burden.
3Device complexity
If high-frequency components are combined through simple addition, then processing simplicity is maintained, but noise reduction accuracy deteriorates
Solution Approach 1:
Instead of simple addition, the system uses weighted addition with specific coefficients (α and β) to combine the high-frequency components H1 and H2-1. This parameter change in the combination method allows for optimized noise reduction performance while maintaining reasonable processing complexity. The weighted approach enables fine-tuning of the contribution from each component to achieve better noise suppression.
Data Source
AI summary
A filter unit decomposes image data from an image sensor adapted to capture images at a predetermined frame rate into a low-frequency component and a first high-frequency component. A motion vector estimation unit reads data from an event sensor, adapted to asynchronously output information on a pixel in which a brightness changes, at a frame rate higher than the predetermined frame rate and estimates a motion vector. A motion compensation unit performs motion compensation based on the motion vector. The filter unit generates a third high-frequency component by adding the first high-frequency component and a second high-frequency component extracted from an image obtained by motion compensation at a predetermined ratio, and reduces a noise in the image data by adding the low-frequency component and the third high-frequency component.