Synthetic Data Generation for Database Schema Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database planning often inaccurately predicts future growth due to misinterpretation of sample data characteristics, leading to biased database and index designs that overemphasize outliers.
Innovation Solution
Generating synthetic data based on stored numeric distribution information within a schema, which is compared to actual data to ensure statistical similarity and adjust the schema accordingly, improving database performance and fraud detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If sample data is used for database planning, then database design can be performed, but the design becomes biased by outliers in the sample data
Solution Approach 1:
The patent creates synthetic data that copies the statistical distribution characteristics of actual data without copying the specific outlier values. By generating artificial datasets that replicate the underlying probability distributions (e.g., normal, log-normal, power-law distributions), the system enables database planning with representative data that excludes spurious outliers present in small sample sets.
2Productivity
If small sample data is used for modeling, then database planning can proceed quickly, but the statistical representation becomes inaccurate
Solution Approach 1:
The patent transforms the input data by adjusting statistical parameters such as mean, standard deviation, skewness, and kurtosis to match the underlying distribution characteristics of the actual data. By modifying these parameters, the system generates synthetic datasets with improved statistical reliability while maintaining the planning speed advantage of using generated data rather than waiting for large sample collections.
3Measurement precision
If actual data is used directly for database indexing, then real-world patterns are captured, but outliers skew the index design
Solution Approach 1:
The patent introduces synthetic data as an intermediary between the actual data and the database indexing process. The synthetic datasets serve as a mediating layer that filters out the harmful influence of outliers while preserving the essential distribution patterns. Database planners can use this intermediary representation to design indexes that reflect true data patterns without being skewed by extreme values in the original sample data.
Data Source
AI summary
A system, method, and computer-readable medium for generating synthetic data are described. Improved data models for databases may be achieved by improving the quality of synthetic data upon for modeling those databases. According to some aspects, these and other benefits may be achieved by using numeric distribution information in a schema describing one or more numeric fields and, based on that schema, distribution-appropriate numerical data may be generated. The schema may be compared against actual data and the schema adjusted to more closely match the actual data. In implementation, this may be effected by storing a schema with distribution information and/or one or more parameters, generating synthetic numerical data based on the schema, and, based on a comparison with actual data, modify the schema until the synthetic data is statistically similar to the actual data. A benefit may include improved database performance and indexing based on repeatable, statistically appropriate, synthetic data.