Cycle GAN RGB to Non-RGB Image Transformation for Autonomous Driving
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for autonomous vehicles face challenges in processing test images in non-RGB formats due to the need for real-time format conversion, which is inefficient and requires extensive training data, especially when converting complex or non-existent formats, and re-learning CNN parameters.
Innovation Solution
A method using a cycle GAN to transform annotated RGB images into non-RGB images, incorporating a learning device with transformers and discriminators to achieve format conversion while optimizing the transformation process and reducing annotation costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If real-time format conversion is performed during testing processes, then the CNN can process non-RGB format images, but computational overhead increases significantly
Solution Approach 1:
The patent applies preliminary action by performing format transformation during the training phase rather than during real-time testing. The cycle GAN is trained to transform RGB images to non-RGB formats and back, creating pre-learned transformation capabilities. This allows the CNN to directly process non-RGB format images during testing without performing computationally intensive real-time conversion, thus resolving the contradiction between format adaptability and processing speed.
2Adaptability or versatility
If the CNN is re-trained with new training images in non-RGB format, then it can process those formats properly, but tremendous time and money are required to generate tens of thousands of training images and GTs
Solution Approach 1:
The patent uses copying by generating synthetic non-RGB format training images through the cycle GAN transformation process. Instead of manually creating tens of thousands of annotated non-RGB images, the system copies and transforms existing RGB training images into non-RGB formats using the learned transformation model. This dramatically reduces the time and resources required for training data preparation while maintaining the ability to process non-RGB formats effectively.
3Productivity
If simple format conversion rules are applied (e.g., YUV to RGB), then conversion overhead is trivial, but complex conversion rules or non-existent rules cannot be applied
Solution Approach 1:
The patent applies parameter changes by using the cycle GAN to learn and adapt transformation parameters for converting between different image formats. The GAN model adjusts its internal parameters during training to capture the specific characteristics of different format conversions, enabling efficient transformation even for complex or non-standard format rules that cannot be handled by simple predefined conversion algorithms.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for learning transformation of an annotated RGB image into an annotated Non-RGB image, in target color space, by using a cycle GAN and for domain adaptation capable of reducing annotation cost and optimizing customer requirements is provided. The method includes steps of: a learning device transforming a first image in an RGB format to a second image in a non-RGB format, determining whether the second image has a primary or a secondary non-RGB format, and transforming the second image to a third image in the RGB format; transforming a fourth image in the non-RGB format to a fifth image in the RGB format, determining whether the fifth image has a primary RGB format or a secondary RGB format, and transforming the fifth image to a sixth image in the non-RGB format. Further, by the method, training data can be generated even with virtual driving environments.