ADAS Camera Reflection Removal Using GAN Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for reflection removal in ADAS camera images are inadequate due to impaired field of view, reliance on additional contextual information, and poor performance with non-blurred reflections, particularly in vehicle windows, leading to suboptimal image quality for ADAS processing.
Innovation Solution
A Generative Adversarial Network (GAN) based training method using two ADAS cameras with and without reflection filters to learn reflection removal without additional contextual information, generating diverse synthetic reflections to improve accuracy and adapt to non-blurred reflections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If mechanical methods are used to remove reflection by placing plastic around the camera, then reflection removal is achieved, but the field of view is impaired
Solution Approach 1:
The patent replaces mechanical reflection removal methods (plastic barriers around the camera) with a computational approach using deep learning algorithms. The system processes images through neural networks to identify and remove reflections digitally, eliminating the need for physical modifications that would block the camera's field of view.
Solution Approach 2:
The patent introduces an intermediate processing layer between the camera and the final image output. A deep learning model acts as an intermediary that analyzes the captured image, separates reflection components from the actual scene, and reconstructs the clean image without requiring physical intervention in the optical path.
2Measurement precision
If additional contextual information is used for reflection removal, then reflection removal accuracy is improved, but the system complexity increases
Solution Approach 1:
The patent enables the system to be self-sufficient by training the deep learning model on diverse synthetic reflection data during the offline phase. Once trained, the model can independently handle various reflection scenarios without requiring additional contextual information or multiple sensors during runtime, thus maintaining accuracy while minimizing system complexity.
Solution Approach 2:
The patent performs preliminary training of the deep learning model offline using synthetically generated reflection data. This preliminary action prepares the model to handle various reflection conditions independently during runtime, eliminating the need for additional contextual information collection and processing during actual operation.
3Extent of automation
If existing deep learning methods are used for reflection removal, then automation is improved, but performance is poor with non-blurred reflections in vehicle windows
Solution Approach 1:
The patent modifies the training parameters and data characteristics by generating synthetic reflections with varying blur levels, including non-blurred reflections specific to vehicle window scenarios. This parameter change in the training data enables the automated system to reliably handle non-blurred reflections that existing methods fail to process correctly.
Solution Approach 2:
The patent creates a universal deep learning model trained on diverse synthetic reflection data that covers multiple reflection types (blurred, non-blurred, various intensities and angles). This universal model can automatically handle different reflection scenarios in vehicle windows without requiring separate specialized systems, improving both automation and reliability.
Data Source
Figure 1
Figure 2
Figure 3.1
AI summary
This invention refers to a method of reflection removal based on a Generative Adversarial Network used for training of an ADAS camera of a vehicle, comprising an acquisition step, a training step and an inference step. In the acquisition step two identical ADAS cameras capture images having essentially the same content. The first camera captures images without reflection and sends them to a first images dataset while the second camera capturing images with natural reflection and sends them to a second images dataset. In a first training step, a data processing hardware acquires two randomly sampled pair of images from the first image dataset and carries out two simultaneous altering and overlapping of the pair of images generating respectively a first mixed image as a sum of the first transmission image with a first synthetical reflection and a second mixed image as a sum of the second transmission image with a second synthetical reflection. In a second training step, the first image together with a third mixed image are altered using third augmentation parameters, the third mixed image proceeding from the second dataset, that is being naturally mixed with real reflection. The output of the first and of the second training steps enter the third training step which is carried out by using the Generative Adversarial Network GAN. In a third training step, based on a machine learning model, the Generator generates a first predicted transmission image corresponding to the first transmission image, a second predicted transmission image corresponding to the second transmission image, and a third predicted transmission image corresponding to the third transmission image; then the machine learning model is optimized for the generation of predicted transmission images as close as possible to the respective transmission images, compressed and sent to a GAN machine learning block. In the inference step, the ADAS camera acquires a single image containing reflection, its GAN machine learning block generates a predicted transmission image, having the reflection suppressed and makes the predicted transmission image available to an ADAS processing chain.