GAN Virtual Feature Maps for Domain Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods for training Deep Convolutional Neural Networks (CNNs) in autonomous vehicles require costly manual acquisition of real-world training images and their ground truths, and alternative virtual approaches lead to performance degradation when detecting real-world objects with different characteristics.
Innovation Solution
A Generative Adversarial Network (GAN) is used to generate virtual feature maps with characteristics similar to real feature maps, comprising a generating network, a discriminating network, an object detection network, and loss units that learn parameters to deceive the discriminator and improve object detection in real-world scenarios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If virtual images are used for training CNNs, then training cost is reduced, but detection performance on real-world objects degrades
Solution Approach 1:
The patent uses a Generative Adversarial Network (GAN) to create virtual images that copy the characteristics of real-world images. The generator network learns to produce synthetic images that are indistinguishable from real images, while the discriminator network attempts to differentiate between real and generated images. This copying approach allows training on virtual data while maintaining real-world fidelity, resolving the contradiction between low training cost and high detection performance.
Solution Approach 2:
The patent transforms real images into virtual feature maps by modifying parameters such as color space (converting RGB to grayscale or other representations), resolution, and feature extraction parameters. This parameter transformation allows the network to learn from virtual representations while maintaining the essential characteristics needed for real-world detection, bridging the gap between virtual training and real performance.
2Measurement precision
If manual acquisition of real-world training images and GTs is performed, then detection accuracy is improved, but training cost increases
Solution Approach 1:
Instead of manually acquiring real images and their ground truths, the patent uses a GAN to copy and generate synthetic images with automatically generated ground truths. The generator creates images that replicate real-world characteristics, while the system automatically generates corresponding annotation data, eliminating the need for expensive manual annotation while maintaining detection accuracy.
Solution Approach 2:
The system performs self-service by automatically generating both training images and their corresponding ground truths through the GAN framework. The generator and discriminator networks work together to create self-contained training datasets with inherent label information, removing the dependency on manual human annotation processes and significantly reducing training costs while preserving accuracy.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of learning for deriving virtual feature maps from virtual images, whose characteristics are same as or similar to those of real feature maps derived from real images, by using GAN including a generating network and a discriminating network capable of being applied to domain adaptation is provided to be used in virtual driving environments. The method includes steps of: (a) a learning device instructing the generating network to apply convolutional operations to an input image, to thereby generate a output feature map, whose characteristics are same as or similar to those of the real feature maps; and (b) instructing a loss unit to generate losses by referring to an evaluation score, corresponding to the output feature map, generated by the discriminating network. By the method using a runtime input transformation, a gap between virtuality and reality can be reduced, and annotation costs can be reduced.