Time-Lapse Image Denoising with Center-Peripheral Frame CNNs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing noise removal algorithms for time-lapse images fail to effectively handle rapid changes in images, often distorting information by removing both noise and signal components simultaneously.
Innovation Solution
A convolutional neural network architecture comprising a center frame processing convolutional neural network, peripheral frame processing convolutional neural network, and information combining convolutional neural network, which utilizes past and future frames to process current frames without relying on identical signal components, allowing for effective noise removal while preserving time series information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing noise removal algorithms based on Noise2Noise framework are used, then noise removal performance is improved, but rapid changes in images are distorted and information is lost
Solution Approach 1:
The patent segments the noise removal process into two distinct networks: a center frame processing network that handles the current frame, and a peripheral frame processing network that handles adjacent frames. This segmentation allows each network to specialize in different aspects of noise removal, preserving rapid changes in the center frame while using peripheral frames for noise suppression.
Solution Approach 2:
The patent introduces an information combining network that acts as an intermediary, fusing the output from the center frame processing network and the peripheral frame processing network. This intermediary combines the strengths of both approaches, maintaining temporal information from the center frame while incorporating noise removal capabilities from peripheral frames.
2Reliability
If algorithms based on similarity between adjacent frames are used, then noise removal is effective, but the algorithm cannot respond to rapid changes and may remove actual signal changes
Solution Approach 1:
The patent applies local quality by making different parts of the system handle different aspects of the problem. The center frame processing network focuses on preserving local temporal information and rapid changes in the current frame, while the peripheral frame processing network focuses on noise removal using similarity from adjacent frames. Each part has specialized functionality tailored to its specific role.
Solution Approach 2:
The patent introduces dynamics by making the system adaptable to different types of changes in the image data. The dual-network architecture allows the system to dynamically adjust between preserving rapid changes (when center frame processing dominates) and removing noise through similarity (when peripheral frame processing dominates), depending on the local characteristics of the input data.
3Measurement precision
If low-rank approximation based on SVD is used, then performance is better for rapid changes, but it cannot handle camera shakes and complex noise patterns
Solution Approach 1:
The patent achieves universality by creating a multi-functional system that can handle various types of noise and image conditions. The center frame processing network handles rapid changes and camera shakes, while the peripheral frame processing network handles noise removal through similarity. The information combining network integrates these functions, making the overall system versatile enough to handle multiple types of noise patterns and image variations.
Data Source
AI summary
Apparatus for removing a noise in a time-lapse image includes a center frame processing convolutional neural network, a peripheral frame processing convolutional neural network and an information combining convolutional neural network. The center frame processing convolutional neural network is configured to receive a current frame of an input image, and output a center processing value. The peripheral frame processing convolutional neural network is configured to receive a plurality of past frames adjacent to the current frame and a plurality of future frames adjacent to the current frame, and output a peripheral processing value. The information combining convolutional neural network is configured to calculate the center processing value and the peripheral processing value, and output an output image. The center frame processing convolutional neural network does not refer to a center pixel of the current frame.