Neural Network Inversion for Synthetic Training Data Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The generation of labeled training data for machine learning models, particularly in supervised learning, is labor and cost intensive, limiting the efficiency of model development and deployment.
Innovation Solution
The method involves combining network inversion with a partially trained classification layer of an artificial neural network (ANN) classifier, using an autoencoder or generative adversarial network (GAN) to generate additional training data, which is then used to further train the classification layer, reducing the reliance on extensive initial training datasets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If vast quantities of labeled training data are used to achieve good machine learning performance, then classification accuracy is improved, but labor and cost increase significantly
Solution Approach 1:
The patent applies preliminary action by using a pre-trained autoencoder to extract latent representations from input data before the classification stage. This preliminary feature extraction creates a compressed representation that captures essential patterns, allowing the classification layer to work more efficiently with fewer labeled examples. The autoencoder is trained beforehand on unlabeled data, performing the labor-intensive feature learning task in advance.
Solution Approach 2:
The patent introduces an intermediary component - the autoencoder's latent space - between the raw input data and the classification layer. This intermediary transforms high-dimensional input data into a lower-dimensional latent representation that preserves important structures while removing redundancy. The latent representations serve as a bridge that enables the classification layer to learn more effectively from limited labeled training data.
2Manufacturing precision
If more labeled training data is generated to improve model performance, then decision boundary accuracy is improved, but time and resources required for data labeling increase
Solution Approach 1:
The patent applies copying by generating synthetic training examples through the latent space of the pre-trained autoencoder. Instead of manually creating or collecting real labeled data, the system copies the essential structures and patterns learned by the autoencoder and generates artificial data points that mimic real data characteristics. These synthetic examples can be generated automatically without manual labeling, yet still improve decision boundary accuracy when used for training the classification layer.
Data Source
AI summary
A method of machine learning model development includes building an autoencoder including an encoder trained to map an input into a latent representation, and a decoder trained to map the latent representation to a reconstruction of the input. The method includes building an artificial neural network classifier including the encoder, and a classification layer partially trained to perform a classification in which a class to which the input belongs is predicted based on the latent representation. Neural network inversion is applied to the classification layer to find inverted latent representations within a decision boundary between classes in which a result of the classification is ambiguous, and inverted inputs are obtained from the inverted latent representations. Each inverted input is labeled with a class that is its ground truth, and thereby producing added training data for the classification, and the classification layer is further trained using the added training data.