Blind-Spot Neural Network Branches for Noise Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional neural networks for image restoration require clean data for training, which is difficult or impossible to obtain in cases like astronomical or medical imaging, and previous attempts at blind-spot networks are inefficient due to information leakage between pixels.
Innovation Solution
A blind-spot neural network architecture with multiple branches that extend the receptive field in different directions, excluding information from the central input pixel, using rotated convolution kernels and composition layers to generate a composite feature volume for noise reduction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If multiple layers within the neural network are used to increase complexity, then the network's ability to process information improves, but information from pixels leaks into neighboring pixels, compromising the blind-spot constraint
Solution Approach 1:
The neural network is divided into multiple independent branches, each processing information from different spatial directions. This segmentation prevents information leakage between pixels while maintaining network complexity, as each branch operates independently with its own convolutional layers and attention mechanisms.
Solution Approach 2:
A composition layer is introduced as an intermediary component that aggregates outputs from multiple blind-spot branches. This composition layer combines the processed information from different directions while maintaining the blind-spot constraint, allowing complex processing without direct pixel-to-pixel information leakage.
2Area of stationary object
If the receptive field is extended in all directions, then more contextual information is available for processing, but the central pixel's information cannot be excluded
Solution Approach 1:
The receptive field is designed with asymmetric masking patterns where the central pixel is excluded while surrounding pixels are included. Different branches use different asymmetric masks oriented in different directions, allowing extensive receptive fields that systematically exclude the central pixel through rotational symmetry of asymmetric patterns.
Solution Approach 2:
The solution moves from a single 2D receptive field to a 3D feature space by stacking multiple 2D branches with different orientations. This dimensional transformation allows the network to access information from all spatial directions while maintaining central pixel exclusion in each 2D plane through the composition layer.
3Reliability
If masking is applied to exclude the central pixel, then the blind-spot constraint is maintained, but the quality of the de-noised image deteriorates
Solution Approach 1:
Multiple blind-spot branches process the same input through different computational paths and feature transformations. By merging their outputs in the composition layer, the system recovers information that would be lost through simple masking, improving image quality while maintaining the blind-spot constraint through the aggregated multi-directional processing.
Data Source
AI summary
A neural network architecture is disclosed for restoring noisy data. The neural network is a blind-spot network that can be trained according to a self-supervised framework. In an embodiment, the blind-spot network includes a plurality of network branches. Each network branch processes a version of the input data using one or more layers associated with kernels that have a receptive field that extends in a particular half-plane relative to the output value. In one embodiment, the versions of the input data are offset in a particular direction and the convolution kernels are rotated to correspond to the particular direction of the associated network branch. In another embodiment, the versions of the input data are rotated and the convolution kernel is the same for each network branch. The outputs of the network branches are composited to de-noise the image. In some embodiments, Bayesian filtering is performed to de-noise the input data.