Self-Supervised Image Restoration via Deep Image Prior
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image restoration methods using neural networks require large datasets of degraded and clean image pairs, which are difficult to obtain in real-world scenarios with complex scenes, moving cameras, dynamic objects, and changing lighting conditions.
Innovation Solution
The method involves capturing degraded images with a first sensor, processing them using an untrained convolutional neural network via the Deep Image Prior approach to generate clean images, creating image pairs, and training a machine learning model to convert degraded images into restored images, which can then be applied to new environments using a second sensor.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If neural networks are used to restore and enhance images, then image restoration quality is improved, but the requirement for large datasets of degraded and clean image pairs increases data collection difficulty
Solution Approach 1:
The system uses the untrained convolutional neural network to automatically generate clean images from degraded images without requiring manual annotation or external clean image sources. The network serves itself by using its own structure and the degraded images to produce the training data, eliminating the need for difficult data collection processes
Solution Approach 2:
The method performs preliminary data generation by creating synthetic clean images from degraded images before the actual training process. This preliminary action of generating training pairs on-demand allows the system to prepare training data without relying on difficult-to-obtain real-world paired datasets
2Measurement precision
If supervised learning is used to train machine learning models with large datasets, then model accuracy is improved, but training time and computational resources increase
Solution Approach 1:
The system creates synthetic copies of clean images by processing degraded images through an untrained convolutional neural network. These copied clean images serve as training targets, allowing supervised learning to proceed with synthetically generated data rather than requiring extensive collection of authentic clean images, thereby reducing training time while maintaining model accuracy
3Adaptability or versatility
If real-world scenarios with complex scenes and dynamic conditions are processed, then applicability and versatility are improved, but the difficulty of obtaining clean-degraded image pairs increases
Solution Approach 1:
The convolutional neural network automatically generates the required clean image counterparts for degraded images from complex real-world scenes without external assistance. This self-service capability enables the system to handle diverse real-world scenarios including complex scenes, moving cameras, and dynamic objects while eliminating the need for difficult data collection processes
Solution Approach 2:
The system creates a universal training data generation process that works across various real-world scenarios including complex scenes, moving cameras, dynamic objects, and changing lighting conditions. The same untrained convolutional neural network approach is applied universally to generate training pairs for all these different conditions, making the system highly adaptable without requiring scenario-specific data collection methods
Data Source
AI summary
A method for processing images comprising: capturing a plurality of degraded images of a first real-world environment with a first sensor; processing each degraded image with a first, untrained convolutional neural network, via a Deep Image Prior approach, to obtain a plurality of clean images, wherein each clean image corresponds to a degraded image; pairing each clean image with its corresponding degraded image to create a plurality of degraded/clean image pairs; training, via a supervised learning approach, a machine learning model to learn a function for converting degraded images into restored images based on the plurality of degraded/clean image pairs; capturing a second plurality of degraded images of a second real-world environment; and using the trained machine learning model to convert the second plurality of degraded images into restored images based on the learned function.