Emulated Dataset Training for Sensitive Data Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large machine learning models face challenges in managing sensitive data, ensuring responsible data usage, and efficiently addressing data defects such as bias, toxicity, and intellectual property issues, particularly due to the complexity and scale of training datasets.
Innovation Solution
The use of emulated datasets generated from abstract descriptions of initial datasets, combined with techniques like compartmentalization and differential privacy, allows for training models without direct exposure to sensitive data, ensuring data separation and minimizing the impact of data disgorgement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If large machine learning models are trained on extensive datasets, then model capability and performance are improved, but the ability to manually inspect and control each training datum deteriorates
Solution Approach 1:
The patent segments the training process into two distinct phases: (1) training a first machine learning model on the extensive dataset to learn data characteristics, and (2) using this first model to generate or filter a second dataset for training the final model. This segmentation enables controlled data curation at scale by automating the inspection and selection process through the intermediary first model, thereby maintaining reliability while preserving operational control.
2Reliability
If sophisticated controls are implemented on training data, then data quality is improved, but the complexity of data curation processes increases
Solution Approach 1:
The patent implements self-service by training a first machine learning model to automatically analyze, evaluate, and curate training data without requiring manual intervention for each datum. This first model serves itself and the data curation process by learning data characteristics from the extensive dataset and applying this knowledge to generate or filter the second dataset, thereby maintaining high data quality while reducing curation complexity through automation.
3Measurement precision
If the volume of training data is increased, then model accuracy is improved, but the time required to remediate defects in the training corpus deteriorates
Solution Approach 1:
The patent applies preliminary action by training the first machine learning model in advance to learn the characteristics and defects of the extensive dataset before the final model training begins. This first model then pre-processes or pre-filters the data to create a cleaned second dataset, thereby identifying and remediating defects beforehand. This preliminary defect remediation through the first model significantly reduces the time required for corrections while maintaining high model accuracy through extensive training data.
Data Source
AI summary
Systems and methods for training machine learning models using emulated datasets are provided. In some instances, it may be undesirable for certain portions of an initial training dataset to be used to train a machine learning model. In other instances, a model may be trained using an initial training dataset and it may be desired to cause the machine learning model to “forget” certain aspects of the training. To address both of these scenarios, a generative machine learning model may be used to generate an emulated training dataset that is based on the initial training dataset. The emulated training dataset is then used to train the machine learning model that is ultimately used to perform downstream tasks. This allows the machine learning model to still be trained to perform the task, without the risk of the machine learning model being exposed to the portions of the training data that are undesired to be used for training.