Synthetic Metrics Data Generation via Frequency Domain Transform
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AIOps tools struggle to generate high-fidelity synthetic metrics data efficiently, which is crucial for accurate AI model training and performance testing, especially in terms of capturing real-world anomalies and correlation features like logs and traces.
Innovation Solution
A computer-implemented method that involves monitoring a target system to collect data metrics, pre-processing these metrics as seeds based on predetermined policies, encoding them using transforms like Fourier or Wavelet transforms, post-processing in the frequency domain, and generating synthetic metrics data by applying inverse transforms, while also capturing labels, values, logs, and traces for correlation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional tools are used to generate synthetic metrics data, then the process is simpler, but the fidelity and accuracy of the generated data deteriorates
Solution Approach 1:
The patent uses Generative Adversarial Networks (GANs) to create synthetic copies of real metrics data that preserve statistical properties, anomaly patterns, and correlation structures. The generator network learns to replicate the distribution and characteristics of real system metrics, producing high-fidelity synthetic data without copying sensitive information directly
Solution Approach 2:
The patent transforms real metrics data through multiple processing stages including normalization, feature extraction, and statistical parameter adjustment. The GAN model learns optimal parameter transformations to generate realistic synthetic data while controlling for fidelity through loss function design and training parameter optimization
2Reliability
If real system data is used for AI model training, then the training accuracy is improved, but the cost of setting up and maintaining the environment increases
Solution Approach 1:
The patent creates synthetic copies of real system metrics data that capture essential characteristics, anomaly patterns, and statistical properties. These synthetic copies serve as substitutes for real data in AI model training, eliminating the need to maintain complex production environments while preserving training effectiveness
Solution Approach 2:
The system uses the organization's own historical metrics data to generate synthetic training data, turning existing data assets into a self-sufficient training resource. This eliminates dependency on external environments or additional hardware while leveraging proprietary data for model development
3Loss of information
If existing tools generate synthetic data, then the process is faster, but the ability to capture real-world anomalies and correlation features deteriorates
Solution Approach 1:
The patent employs adversarial feedback mechanisms where the discriminator network continuously evaluates generated data against real data, providing feedback that guides the generator to improve anomaly capture and correlation preservation. This iterative feedback loop ensures high-fidelity reproduction of complex data patterns
Solution Approach 2:
The patent performs preliminary analysis of real metrics data to identify and preserve critical anomaly patterns, correlation structures, and statistical properties before generation. Feature extraction and statistical characterization are conducted in advance to guide the synthetic data generation process
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables the generation of high-fidelity synthetic metrics data that accurately reflects real-world system behavior, including anomalies and correlation features, thereby enhancing the effectiveness of AI model training and performance testing without the high cost of setting up an environment.
Implementation Method 1
encoding, by the processor set, the pre-processed seed using a transform
Implementation Method 2
encoding, by the processor set, the pre-processed seed using a transform
Implementation Method 3
generating, by the processor set, synthetic metrics data by applying an inverse transform to the post-processed seed
Implementation Method 4
generating, by the processor set, synthetic metrics data by applying an inverse transform to the post-processed seed
Data Source
AI summary
Embodiments monitor a target system to collect at least one data metric; pre-process the at least one data metric as a seed based on a predetermined policy; encode the pre-processed seed using a transform; post-process the encoded seed in a frequency domain; generate synthetic metrics data by applying an inverse transform to the post-processed seed; and train an artificial intelligence (AI) model using the generated synthetic metrics data.