ADAS Camera Reflection Suppression Using GAN Training Data Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for reflection removal in ADAS camera images are inadequate due to the need for additional contextual information, impaired field of view, and inability to handle non-blurred reflections common in vehicle windows, particularly affecting image quality and processing accuracy.
Innovation Solution
A method using Generative Adversarial Networks (GANs) trains ADAS cameras by capturing images from two identical cameras with and without reflections, employing data augmentation and reflection synthesis to generate diverse reflection scenarios, optimizing a GAN model for reflection suppression without requiring additional contextual data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If mechanical reflection removal filters are placed around the camera, then reflection is removed, but the field of view is impaired
Solution Approach 1:
The patent replaces mechanical reflection removal filters with a software-based deep learning approach. A neural network model processes the captured images to identify and remove reflections computationally, eliminating the need for physical filters that would block or distort the optical path and reduce the field of view.
Solution Approach 2:
The patent introduces an intermediate processing step using a deep learning model that acts as a mediator between the captured image and the final output. The model learns to separate reflection components from the actual scene, providing a software-based solution that preserves the full field of view while removing harmful reflections.
2Measurement precision
If additional contextual information is used for reflection removal, then reflection removal accuracy is improved, but device complexity increases
Solution Approach 1:
The patent implements a self-service approach where the system captures multiple images from different viewpoints and uses these images to train and apply a deep learning model for reflection removal. The system serves itself by using its own captured data to improve its performance without requiring external contextual information or additional sensors.
Solution Approach 2:
The patent transitions from 2D image processing to utilizing the third dimension of time by capturing images at different time points and viewpoints. This temporal and spatial dimensionality allows the deep learning model to learn reflection patterns and remove them accurately without requiring additional contextual sensors or complex hardware.
3Measurement precision
If deep learning methods are used for reflection removal, then processing accuracy is improved, but computational requirements and training complexity increase
Solution Approach 1:
The patent performs preliminary action by capturing a set of images from different viewpoints in advance, which are then used to train the deep learning model. This pre-capture phase creates a training dataset that enables the model to learn reflection patterns, allowing for accurate reflection removal during actual operation without requiring complex real-time processing or additional hardware.
Data Source
AI summary
A method of reflection removal based on a GAN used for training of an ADAS vehicle camera includes acquisition, training and inference operations. In a first training operation, data processing hardware acquires two randomly sampled pairs of images from a first image dataset and carries out two simultaneous altering and overlapping of the image pair generating first and second mixed images. In a second training operation, the first image together with a third mixed image are altered, the third mixed image proceeding from a second dataset. The output of the first and second training operations enter the third training operation which is carried out by using the GAN. A third training operation generates first, second and third predicted transmission images; and a machine learning model is optimized for the predicted transmission images as close as possible, compressed and sent to a GAN machine learning block.