Camera Lens Contamination Detection Using CNN and Edge Visibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing contamination detection systems for camera lenses in autonomous driving systems face challenges in accurately detecting transparent and translucent contaminants, especially in regions with small pixel differences such as the sky or road surface, and are prone to inaccuracies due to varying light sources and monotonous patterns.
Innovation Solution
A method using a Convolutional Neural Network (CNN) trained with various contamination patterns to classify contamination states in real-time, combined with edge maps generated from pixel brightness differences and external sensor data to determine contamination on camera lenses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If traditional contamination detection methods comparing image visibility to normal threshold values are used, then the detection process is simple, but the accuracy decreases for regions such as sky, ground, or flat objects that have similar patterns to contamination patterns
Solution Approach 1:
The patent segments the detection process into multiple specialized modules: a visibility calculating unit that generates edge maps to assess overall image clarity, and a CNN-based contamination calculating unit that analyzes specific contamination patterns. This segmentation allows each module to specialize in different aspects of detection, improving overall accuracy without excessive complexity.
Solution Approach 2:
The patent introduces an intermediary credibility evaluation mechanism that assesses whether the input image is suitable for contamination detection based on visibility metrics. This intermediary step filters out images with monotonous patterns (like sky or road surfaces) before they reach the contamination analysis stage, preventing false detections while maintaining system efficiency.
2Reliability
If contamination detection is performed by comparing images taken prior to and after headlight exposure, then translucent contamination can be detected, but accuracy decreases when images produce different results under different light sources
Solution Approach 1:
The system uses the camera's own captured images across different time points and lighting conditions as input, eliminating the need for external reference images or controlled lighting environments. The CNN model learns to identify contamination patterns that are consistent across varying conditions, making the system self-sufficient and adaptable to real-world variability.
Solution Approach 2:
The patent leverages natural changes in lighting parameters (headlight activation, ambient light variations) and temporal parameters (multiple time points) as input variations to the CNN model. The model learns to distinguish contamination from normal scene variations by analyzing how different parameters affect the image, improving robustness against lighting changes.
3Adaptability or versatility
If CNN is used to extract features from consecutive frames for contamination detection, then various contamination patterns can be identified, but accuracy decreases in local areas or specific areas such as ground or sky when the CNN has not been trained with various contamination patterns
Solution Approach 1:
The patent performs preliminary credibility evaluation of input images before submitting them to the CNN for contamination analysis. This preliminary action filters out images with monotonous patterns (sky, road surfaces) that would cause the CNN to produce inaccurate results, ensuring that only images with sufficient visual information and contamination likelihood are analyzed, thereby maintaining high local area detection accuracy.
Solution Approach 2:
The system applies different processing qualities to different regions: the credibility evaluation focuses on overall image suitability, while the CNN performs detailed local contamination pattern recognition. This local quality approach ensures that computational resources are concentrated on regions where contamination detection is both possible and necessary, improving local area accuracy.
4Measurement precision
If pixel movement measurement is used to determine contamination, then contamination on specific areas can be detected, but the optical flow is difficult to analyze in environments where pixel differences are not large, such as the sky or road surface
Solution Approach 1:
The patent replaces the optical flow analysis mechanism (which relies on pixel movement and is difficult in monotonous regions) with a CNN-based contamination pattern recognition system. The CNN directly analyzes spatial patterns and textures in the image to identify contamination, eliminating the need for complex optical flow calculations and working effectively in regions with small pixel differences.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for determining whether contamination exists on a camera lens in real-time is provided. The method includes steps of: (a) determining whether a credibility indicating a degree of suitability of an input image being used for detecting the contamination on the camera lens satisfies at least one preset condition; (b) classifying each of contamination states for each of sub-regions by applying CNN operations to generate each degree of frame contamination for each of the sub-regions, and generating each edge map for each of the sub-regions by referring to differences in brightness between each of pixels and its neighboring pixels to generate each degree of frame visibility for each of the sub-regions; and (c) determining whether the contamination exists on the camera lens by referring to the degree of frame contamination and the degree of frame visibility.