Dual GAN Robot Control Server for Sim-to-Real Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current domain adaptation methods, such as RCAN, face performance degradation due to a visual gap between real and simulated images, requiring large amounts of data and extended learning periods, especially when fewer image data are available.
Innovation Solution
The use of two Generative Adversarial Networks (GANs) to generate real-like-fake images from simulated images and canonical images from real images, allowing for more accurate robot control with a reduced number of real images by learning and generating canonical images for robot operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If RCAN is used to convert simulated images to canonical images, then robot control accuracy is improved, but a large amount of training data (millions of units) is required and learning time increases
Solution Approach 1:
The patent segments the single RCAN model into two separate GAN models: a first GAN that converts simulated images to real-like images, and a second GAN that converts real-like images to canonical images. This segmentation allows each model to be trained on smaller, more targeted datasets rather than requiring millions of training images simultaneously, thereby reducing the overall training data volume while maintaining control accuracy.
Solution Approach 2:
The patent introduces real-like images as an intermediary representation between simulated images and canonical images. The first GAN generates real-like images from simulated images, and the second GAN converts these real-like images to canonical images. This intermediary approach enables the system to bridge the visual gap without requiring direct training on large amounts of real-world data, thus reducing training data requirements while preserving accuracy.
2Measurement precision
If RCAN is used to convert simulated images to canonical images, then robot control accuracy is improved, but learning period increases
Solution Approach 1:
By dividing the learning task into two separate GAN models with distinct training objectives, the patent enables parallel or sequential training that converges faster than training a single complex RCAN model. The first GAN focuses on sim-to-real translation while the second GAN focuses on real-to-canonical translation, allowing each to learn more efficiently and reduce overall learning time.
Solution Approach 2:
The first GAN performs preliminary action by converting simulated images to real-like images before the second GAN processes them to canonical images. This preliminary transformation prepares the data in a more suitable format for the second stage, enabling faster convergence and reducing the total learning period required to achieve accurate robot control.
3Ease of manufacture
If simulated images are used for robot training, then data acquisition cost is reduced, but visual gap between simulated and real images decreases control accuracy
Solution Approach 1:
The patent uses real-like images generated by the first GAN as an intermediary that bridges the visual gap between simulated and real images. This intermediary representation captures realistic visual characteristics while being derived from simulated images, allowing the system to maintain the ease of simulated data generation while achieving the visual fidelity needed for accurate robot control.
Solution Approach 2:
The first GAN creates copies of simulated images that resemble real images by learning the mapping between simulated and real image distributions. These copied real-like images preserve the ease of generation from simulated data while acquiring the visual properties necessary for accurate robot control, thus resolving the contradiction between data acquisition ease and control accuracy.
Data Source
AI summary
Disclosed herein are a control server and method for controlling a robot using an artificial neural network, and a robot implementing the same. The robot includes a driver, a processor controlling the driver using an artificial neural network-based algorithm model, and a learning processor learning the algorithm model, where the algorithm model includes a first GAN (generative adversarial network) and a second GAN, the processor generates a plurality of real-like-fake images using a first generator included in a learned first GAN, the learning processor learns a second GAN based on a plurality of random texture rendered images and the plurality of real-like-fake images, and the processor generates a canonical image by inputting a real image to a second generator included in a learned second GAN, and controls the driver using the canonical image.