Synthetic Road Image Generation for Atypical AV Training Scenarios

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Obtaining images depicting atypical scenarios, such as emergency situations or unusual road patterns, for training autonomous vehicle models is costly and challenging, as these scenarios are difficult to capture in real life.

Innovation Solution

A method using a generative adversarial network (GAN) to generate synthetic images from user-generated graphical representations, which are indistinguishable from sensor-detected images, allowing for the training of perception models with realistic and diverse scenarios without the need for expensive and hard-to-obtain real images.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If real images depicting atypical scenarios are obtained for training, then the quality and robustness of the perception model is improved, but the cost and time required to obtain training images increases

Engineering Contradiction:

Improveperception model robustnessVSAvoidtime to obtain training images

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent uses generative adversarial networks to create synthetic copies of real-world images depicting atypical scenarios. These synthetic images serve as substitutes for expensive and time-consuming real image captures, while maintaining the visual characteristics needed for training perception models in autonomous vehicles

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The system performs preliminary generation of synthetic training images before actual training needs arise. By pre-generating diverse atypical scenarios including emergency situations and unusual road patterns, the system eliminates the need for time-consuming real image capture when training data is needed

Inventive Principle:

Principle #10Preliminary action

2Reliability

If real images depicting atypical scenarios are obtained for training, then the quality and robustness of the perception model is improved, but the cost to obtain training images increases

Engineering Contradiction:

Improveperception model robustnessVSAvoidcost to obtain training images

Core Design Contradiction:

ReliabilityVSLoss of substance

Solution Approach 1:

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The system generates synthetic images that are computationally inexpensive to create compared to real image capture campaigns. These synthetic training images can be generated on-demand without the substantial costs associated with hiring personnel, equipment, and logistics for capturing real atypical scenarios

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

3Loss of time

If synthetic images are generated using generative networks, then the cost and time to obtain training images is reduced, but the realism and quality of training data may be compromised

Engineering Contradiction:

Improvetime to obtain training imagesVSAvoidimage realism accuracy

Core Design Contradiction:

Loss of timeVSMeasurement precision

Solution Approach 1:

The patent employs a generative adversarial network architecture where a discriminator network provides feedback to the generator network. The discriminator evaluates the realism of generated images and guides the generator to improve, creating a feedback loop that continuously enhances image quality and realism until the synthetic images are indistinguishable from real photographs

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

Inventive Principle:

Principle #10Preliminary action

4Reliability

If a wide range of scenarios are simulated for training, then the robustness of the perception model is improved, but the complexity of the training process increases

Engineering Contradiction:

Improveperception model robustnessVSAvoidtraining process complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The generative adversarial network serves multiple functions simultaneously: it generates diverse atypical scenarios, ensures visual realism through the discriminator, and can produce various types of training data (images, annotations) in a unified framework. This multi-functionality reduces the need for separate systems for each training data requirement

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS11893775B2Navigating a vehicle based on data processing using synthetically generated images

Publication Date: 2024.02.06 PLUSAI INC

AI summary

A user-generated graphical representation can be sent into a generative network to generate a synthetic image of an area including a road, the user-generated graphical representation including at least three different colors and each color from the at least three different colors representing a feature from a plurality of features. A determination can be made that a discrimination network fails to distinguish between the synthetic image and a sensor detected image. The synthetic image can be sent, in response to determining that the discrimination network fails to distinguish between the synthetic image and the sensor-detected image, into an object detector to generate a non-user-generated graphical representation. An objective function can be determined based on a comparison between the user-generated graphical representation and the non-user-generated graphical representation. A perception model can be trained using the synthetic image in response to determining that the objective function is within a predetermined acceptable range.