Synthetic Relational Data Linking for Referential Integrity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods fail to generate synthetic data that accurately replicates the functional relationships between tables in relational datasets, leading to inadequate testing and development of systems that rely on these datasets.
Innovation Solution
Generate extended tables based on a reference relational dataset, including repetitions of data values in parent and child tables, and use these extended tables to determine links between synthetic tables, preserving functional aspects like cardinality and referential integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing methods are used to generate synthetic data for individual tables, then the generation process is simple and fast, but the functional relationships between tables are not preserved
Solution Approach 1:
The patent segments the relational dataset into individual tables while identifying and preserving foreign key relationships. Each table is processed separately to generate synthetic data, but the foreign key constraints are maintained to preserve functional relationships between tables. This segmentation approach allows independent table processing while ensuring relational integrity through foreign key preservation.
2Reliability
If large real-world datasets are used for testing, then comprehensive testing coverage is achieved, but storage and processing resources are excessively consumed
Solution Approach 1:
The patent creates synthetic copies of real-world datasets that replicate the statistical properties, schema structure, and foreign key relationships of the original data. These synthetic copies serve as testing datasets with reduced storage requirements while maintaining comprehensive testing coverage. The copying process preserves functional relationships through foreign key constraints, enabling realistic testing without requiring large volumes of actual sensitive data.
3Reliability
If foreign key relationships are included in synthetic data generation, then referential integrity is maintained, but the generation process becomes more complex and time-consuming
Solution Approach 1:
The patent performs preliminary analysis of the reference dataset to identify all foreign key relationships, parent-child table connections, and referential integrity constraints before generating synthetic data. This preliminary action includes creating a mapping of foreign keys to their corresponding parent tables and relationships. By preparing this structural information in advance, the actual synthetic data generation can proceed efficiently while automatically maintaining referential integrity through the pre-established foreign key mappings.
Data Source
AI summary
Methods and apparatus for generating synthetic data based on a reference relational dataset comprising a plurality of tables and a plurality of relationships between the tables. The methods include generating extended tables for each table in the reference relational dataset and generating synthetic extended tables for each extended table. Measures of similarity are then determined between data records in child and parent synthetic extended tables and links between data records determined based on the measures measure of similarity. Generated synthetic data includes data from the synthetic extended tables and for child tables, a foreign key field populated based on the determined links between data records child and parent synthetic extended tables.