Refining Synthetic Data With A Generative Adversarial Network Using Auxiliary Inputs

Abstract

The present invention extends to methods, systems, and computer program products for refining synthetic data with a Generative Adversarial Network (GAN) using auxiliary inputs. Refined synthetic data can be rendered more realistically than the original synthetic data. Refined synthetic data also retains annotation metadata and labeling metadata used for training of machine learning models. GANs can be extended to use auxiliary channels as inputs to a refiner network to provide hints about increasing the realism of synthetic data. Refinement of synthetic data enhances the use of synthetic data for additional applications.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS[0001]Not applicable.BACKGROUND1. Field of the Invention[0002]This invention relates generally to the field of formulating realistic training data for training machine learning models, and, more particularly, to refining synthetic data with a generative adversarial network using auxiliary inputs.2. Related Art[0003]The process of annotating and labeling relevant portions of image training data (e.g., still images or video) for training machine learning models can be tedious, time-consuming, and expensive. To reduce these annotating and labeling burdens, synthetic data (e.g., virtual images generated by gaming or other graphical engines) can be used. Annotating synthetic data is more straightforward as annotation is a direct by-product of generating the synthetic data.BRIEF DESCRIPTION OF THE DRAWINGS[0004]The specific features, aspects and advantages of the present invention will become better understood with regard to the following description...

AI Technical Summary

Benefits of technology

The patent text describes a method called GAN that uses additional data, such as semantic maps and depth maps, to ensure the correct textures are applied to different parts of a virtual image. This results in a more realistic and accurate image. The technique can also be used to generate refined images for training other models, such as computer vision or autonomous driving systems.

Problems solved by technology

The process of annotating and labeling relevant portions of image training data (e.g., still images or video) for training machine learning models can be tedious, time-consuming, and expensive.

Images