Time Series Evaluation Framework for Generated Data Realism
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods struggle to evaluate the realism of generated training datasets for machine learning algorithms, particularly for technical time series data points, as human intuition is insufficient for assessing abnormalities in such data.
Innovation Solution
An evaluation framework is provided that combines visual analytics with algorithmic methods to assess the quality of generated time series datasets, using multiple views to facilitate user interaction and comparison with real datasets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data generation algorithms (GANs) are used to generate training datasets, then the availability of training data is improved, but the ability to assess the realism/quality of generated data deteriorates
Solution Approach 1:
The patent introduces an evaluation framework as an intermediary system that includes multiple views (iteration view showing distance measures, instance view showing time series instances, selected instance view showing detailed plots) to mediate between the generated data and the user's assessment capability. This framework enables experts to evaluate the realism of generated time series data by providing comprehensive visualizations and comparison capabilities.
Solution Approach 2:
The patent transforms the assessment problem from a single-dimension task to a multi-dimensional evaluation system. It adds multiple dimensions including: different views (iteration view, instance view, selected instance view), multiple distance measures (e.g., Dynamic Time Warping, Euclidean distance), and various plot types (heatmaps, line plots, scatter plots) to enable comprehensive quality assessment of generated data.
2Measurement precision
If multiple evaluation metrics and views are provided, then the assessment capability is improved, but the system complexity increases
Solution Approach 1:
The evaluation framework is segmented into multiple independent views, each serving a specific assessment purpose: iteration view for overall quality assessment, instance view for comparing multiple time series instances, and selected instance view for detailed analysis. This segmentation allows users to access only the evaluation metrics and visualizations relevant to their specific needs, managing complexity through modular organization.
Solution Approach 2:
The evaluation framework is designed as a universal system that can assess different types of time series data (technical, medical, industrial) using the same multi-view structure. The framework provides multiple distance measures and plot types that can be applied across different domains, reducing the need for domain-specific evaluation systems while maintaining comprehensive assessment capability.
Data Source
AI summary
An evaluation framework for a generated dataset of a data generation algorithm such as a generative adversarial network is provided. The generated dataset includes a plurality of iterations of multiple instances of generated time series of data points. The evaluation framework provides multiple views. A first view includes at least one distance measure. The at least one distance measure is between the multiple instances of the generated time series and multiple instances of a reference time series, as a function of the plurality of iterations.