Synthetic Data Generation Using Clustered Gaussian Distributions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The development of machine learning models is hindered by limited training datasets and data points, leading to delays in training and deployment due to insufficient data availability.
Innovation Solution
A computer-implemented method generates synthetic data using a Gaussian distribution, identified clusters, and a binary classifier approach to create statistically correct data points, ensuring consistent data accessibility and enhancing model development efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If synthetic data is generated using Gaussian distribution and clustering, then data quantity is increased, but data quality and statistical accuracy may deteriorate
Solution Approach 1:
The patent implements a feedback mechanism where a binary classifier is trained to distinguish between real and synthetic data points. The classification accuracy is used to quantitatively assess the quality of generated synthetic data. If the classifier can easily distinguish synthetic from real data (indicating poor synthetic data quality), the generation process is adjusted. This closed-loop feedback ensures that synthetic data maintains statistical accuracy and indistinguishability from real data while increasing data quantity.
Solution Approach 2:
The patent employs parameter optimization in the Gaussian distribution generation process, specifically adjusting the covariance matrices for each cluster to match the statistical properties of real data. By carefully tuning these parameters (mean vectors, covariance matrices) based on the clustered real data characteristics, the synthetic data generated maintains statistical fidelity and distributional accuracy while providing increased data quantity.
2Productivity
If clusters are identified using trained classification models, then data generation efficiency is improved, but model complexity and computational requirements increase
Solution Approach 1:
The patent segments the dataset into distinct clusters before generating synthetic data. By identifying clusters in the data space and generating synthetic points cluster-specifically using tailored Gaussian distributions, the method achieves efficient data generation. Each cluster receives customized generation parameters based on its local statistical properties, improving overall efficiency while keeping individual cluster models relatively simple.
Solution Approach 2:
The patent performs preliminary clustering and statistical analysis of the real data before synthetic data generation. The covariance matrices and mean vectors are pre-computed from the clustered real data, and the binary classifier is pre-trained to establish quality benchmarks. This preliminary action prepares all necessary parameters and models in advance, enabling efficient synthetic data generation without requiring complex real-time computations during the generation process.
Data Source
AI summary
In an approach to generating quantitatively assessed synthetic training data, one or more computer processors identify an initial plurality of clusters in a dataset utilizing a trained classification model and a plurality of associated hyperparameters, wherein the clusters have sufficient density to be represented in a calculated probability distribution. The one or more computer processors generate one or more synthetic data points for each identified cluster utilizing a corresponding calculated probability distribution. The one or more computer processors quantitatively assess the one or more generated synthetic data points.