Synthetic Training Data Generation for PII-Safe Database Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing software testing processes utilize real customer data, which poses a risk of exposing personal identifiable information (PII), necessitating a need for secure and efficient synthetic training data generation methods.
Innovation Solution
A system and method for generating synthetic training data using a computer program that extracts entities from user speech, creates a database agnostic ANSI SQL-standard data structure, generates synthetic data, and applies encryption and masking techniques to protect sensitive information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If real customer data is used for software testing, then testing accuracy and realism are improved, but the risk of PII exposure increases
Solution Approach 1:
The patent creates synthetic copies of real customer data that preserve the statistical properties, relationships, and patterns of the original data while containing no actual PII. The synthetic data generation system replicates the structure and characteristics of production data environments without using real customer information, thereby maintaining testing accuracy while eliminating privacy risks
Solution Approach 2:
The patent introduces synthetic data as an intermediary between real customer data and testing processes. Instead of directly using real PII-containing data, the system generates intermediate synthetic representations that maintain the essential characteristics needed for accurate testing while breaking the direct connection to actual customer information
2Reliability
If synthetic data is generated to protect PII, then data security is improved, but data compatibility across different databases may worsen
Solution Approach 1:
The patent creates a universal synthetic data generation system that produces database-agnostic output compatible with multiple database types including SQL and NoSQL databases. The system generates synthetic data in standardized formats that can be adapted to various database schemas and structures, ensuring broad compatibility while maintaining security
Solution Approach 2:
The patent employs parameter-based configuration to adjust synthetic data generation settings according to different database requirements. By changing parameters such as data format, schema structure, and distribution characteristics, the system maintains database compatibility across different platforms while consistently ensuring data security through synthetic generation
3Measurement precision
If entity extraction and data structure creation are performed manually, then data accuracy is improved, but processing time and complexity increase
Solution Approach 1:
The patent implements self-service automated entity extraction using machine learning models that automatically identify and extract entities from input text without manual intervention. The system autonomously performs data structure creation, column identification, and schema generation, eliminating the need for manual data processing while maintaining high accuracy through trained algorithms
Solution Approach 2:
The patent replaces manual mechanical processes of data extraction and structure creation with automated computational systems. Machine learning models and natural language processing algorithms substitute human analysts, performing entity recognition, classification, and data structure generation automatically, thereby reducing processing time while maintaining or improving accuracy
Data Source
AI summary
Systems and methods for generating synthetic training data are disclosed. A method may include: (1) receiving user speech from a user; (2) generating an input file comprising text of the user speech; (3) extracting entities from the text in the input file; (4) creating an input data structure for a data structure for the entities, wherein the input data structure comprises a plurality of columns, a column name for each column, a data attribute for each column, and a data type for each column, and a number of records based on a volume parameter; (5) converting the data type for each column to an ANSI SQL-standard data type; (6) generating a database agnostic data structure having the column names and the ANSI SQL-standard data type; (7) generating synthetic data for the database agnostic data structure; and (8) outputting an output file comprising the synthetic data.