Generative Synthetic Data Model for Drift Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning model performance monitoring techniques rely on original training datasets for data drift detection, which poses challenges in data storage, governance, and increased exposure risks.
Innovation Solution
The use of a generative synthetic data model to replace historical training datasets, allowing for the on-demand generation of synthetic datasets that simulate the historical training data, thereby eliminating the need for storing robust training datasets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If historical training datasets are stored separately from deployed models for data drift detection, then data drift detection accuracy is improved, but data exposure risks and data governance complexities increase
Solution Approach 1:
The patent creates synthetic copies of historical training data through generative models. Instead of storing and exposing actual sensitive training data, the system generates synthetic datasets that replicate the statistical properties and distributions of the original training data. This allows data drift detection to be performed on synthetic copies rather than the real data, maintaining detection accuracy while eliminating data exposure risks.
2Measurement precision
If historical training datasets are stored separately from deployed models, then data drift detection is enabled, but system complexity and data governance requirements increase
Solution Approach 1:
The patent merges the historical training data representation with the deployed model by storing the generative model parameters alongside the model weights. Instead of maintaining separate storage systems for training data and models, the system combines them into a unified structure where the generative model serves as a compact representation of the historical data that travels with the deployed model.
3Reliability
If robust historical training datasets are maintained for data drift detection, then detection reliability is improved, but memory resource consumption increases
Solution Approach 1:
The patent transforms the historical training dataset from a large-scale data structure into a compact parametric representation through the generative model. Instead of storing millions of training samples that consume significant memory, the system stores the parameters and weights of the generative model that can reproduce the data distribution, dramatically reducing memory requirements while preserving the essential characteristics needed for reliable data drift detection.
Data Source
AI summary
Various embodiments of the present disclosure provide machine learning model performance monitoring techniques for automatically generating performance metrics for a machine learning model. The techniques may include identifying a generative synthetic data model corresponding to a historical training dataset for a target machine learning model, generating, using the generative synthetic data model, a synthetic dataset for the target machine learning model, generating a performance output for the target machine learning model based on a comparison between the synthetic dataset and a contemporary input dataset, and initiating the performance of one or more model performance-based operations based on the performance output for the target machine learning model.