Synthetic Image Pair Generation for Neural Network Noise Correction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Automated driving systems face performance degradation due to environmental noise factors like sunlight, rain, and fog, leading to false detections and incomplete measurements, which are not effectively addressed by existing methods.

Innovation Solution

A method for generating digital image pairs using convolutional neural networks to correct noisy image components, where 'clean' training data is created by determining object movement and solid angle consistency between overlapping images from a mobile platform's environment, allowing for noise suppression without ground-truth annotated data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If clean training data is used to train neural networks for noise suppression, then the quality of noise correction is improved, but the cost and difficulty of obtaining such data increases significantly

Engineering Contradiction:

Improvenoise correction qualityVSAvoiddata acquisition difficulty

Core Design Contradiction:

Measurement precisionVSEase of manufacture

Solution Approach 1:

The patent creates synthetic clean images by warping reference images to match the viewpoint and geometry of noisy images. This copying approach generates unlimited clean training data without requiring expensive ground-truth annotations, resolving the contradiction between data quality and acquisition difficulty

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The system performs preliminary actions by pre-processing reference images through viewpoint transformation and geometric warping before pairing them with noisy images. This preliminary preparation creates ready-to-use clean training data that can be automatically generated in advance, eliminating the need for costly manual annotation processes

Inventive Principle:

Principle #10Preliminary action

2Adaptability or versatility

If images are taken from different viewpoints to increase data diversity, then the adaptability of training data is improved, but the consistency between corresponding pixels in noisy and clean images deteriorates

Engineering Contradiction:

Improvetraining data diversityVSAvoidpixel correspondence accuracy

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent dynamically adjusts the viewpoint of reference images to match the noisy image being processed. By transforming the viewpoint of clean reference images to correspond exactly with the noisy image perspective, the system maintains pixel-level correspondence while still using diverse reference images from different original viewpoints, thus resolving the contradiction between data diversity and pixel accuracy

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system applies local geometric transformations and warping to reference images based on their spatial relationship to the noisy image. This local adaptation ensures that each region of the reference image is correctly aligned with its corresponding region in the noisy image, preserving pixel correspondence while accommodating diverse viewpoints

Inventive Principle:

Principle #3Local quality

3Duration of action of moving object

If object movement between images is allowed to increase temporal coverage, then the duration of training data collection is improved, but the accuracy of pixel correspondence between images deteriorates

Engineering Contradiction:

Improvetemporal coverageVSAvoidpixel correspondence accuracy

Core Design Contradiction:

Duration of action of moving objectVSMeasurement precision

Solution Approach 1:

The patent replaces mechanical alignment methods with computational image processing techniques. By using feature detection, optical flow analysis, and geometric warping algorithms, the system can compensate for object movement and maintain pixel correspondence accuracy even when images are captured at different times, thus resolving the contradiction between temporal coverage and alignment precision

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS11321815B2Method for generating digital image pairs as training data for neural networks

Publication Date: 2022.05.03 ROBERT BOSCH GMBH

AI summary

A method for generating a digital image pair for training a neural network to correct noisy image components of noisy images includes determining an extent of object movements within an overlapping region of a stored first digital image and a stored second digital image of an environment of a mobile platform, and determining a respective acquired solid angle of the environment of the mobile platform of the first and second digital images. The method further includes generating the digital image pair from the first digital image and the second digital images, when the respective acquired solid angles of the environment of the first and the second digital image do not differ from one another by more than a defined difference, and the extent of the object movements within the overlapping region of the first and the second digital image is less than a defined value.