Self-Supervised Image Deraining With Pixel-Wise Rain Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing self-supervised learning methods fail to effectively remove rain streaks from images due to non-uniform distribution and dependency on prior knowledge of rain distribution, and deep learning models require large-scale datasets which are expensive and tedious to collect.
Innovation Solution
A self-supervised learning model using a Probability Estimation Network (PEN) to predict pixel-wise rain likelihood, combined with a prior knowledge generation model and self-supervised learning model, to classify rainy and non-rainy pixels, and an image inpainting method to generate derained images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If self-supervised learning methods are used for image deraining, then dependency on large-scale labeled datasets is avoided, but the methods fail to remove rain streaks effectively due to non-uniform distribution of rainy pixels
Solution Approach 1:
The method segments the image into rainy and non-rainy regions using a probability estimation network that classifies each pixel independently. This segmentation approach allows the model to handle non-uniform rain distribution without requiring paired training data, resolving the contradiction between ease of manufacture and reliability.
Solution Approach 2:
A probability estimation network serves as an intermediary component that generates rain probability maps from single unlabeled rainy images. This intermediary enables the subsequent deraining process to work effectively without paired datasets, maintaining both ease of manufacture and reliability.
2Device complexity
If existing self-supervised methods are applied to image deraining, then labeled dataset requirements are reduced, but performance deteriorates due to explicit assumptions about uniform noise distribution
Solution Approach 1:
The method applies local quality by processing each pixel independently through the probability estimation network, allowing the model to adapt to local rain patterns without assuming uniform distribution. This enables high deraining quality while reducing training data requirements.
Solution Approach 2:
The system performs self-service by generating its own training signals from unlabeled rainy images through the probability estimation network. The model learns to identify and remove rain streaks using only single images, eliminating the need for paired datasets while maintaining high deraining quality.
3Reliability
If deep learning models are trained with large-scale datasets, then model performance improves, but data collection becomes expensive and tedious
Solution Approach 1:
The model serves itself by learning from unlabeled rainy images without requiring manual annotation or paired clean images. The probability estimation network automatically generates training signals from the input images themselves, achieving high performance while eliminating time-consuming data collection processes.
Solution Approach 2:
The method creates a probabilistic copy of the rain distribution pattern from the input image itself, using the rain probability map as a surrogate for ground truth labels. This copying mechanism enables effective training without accessing external labeled datasets, reducing both cost and time.
Data Source
AI summary
A method for generating a derained image using a self-supervised learning model is provided. The method includes (i) processing an input image received from a user device; (ii) classifying pixels of the input image as rainy or non-rainy pixels by assigning a first value to the rainy pixels and a second value to the non-rainy pixels; (iii) generating a masked area in the input image by masking out the rainy pixels from the input image based on the rainy pixels; (iv) generating an inpainted image by filling the masked area using an image inpainting method; and (v) generating an optimized image by deraining the inpainted image and remove image artifacts and blurriness caused by the image inpainting method. The optimized image enables an improved semantic segmentation of the input image and object detection.