Cloning Sanitized Data Records for Testing Environments
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The process of provisioning data from a production environment to a testing environment is inefficient and laborious, especially when dealing with large volumes of data that require sanitization to remove nonpublic personal information, as it involves a single point generation and laborious sanitization and integration processes, often taking months to complete.
Innovation Solution
A cloning process that generates multiple sanitized data records by algorithmically replacing nonpublic personal information with fictitious data and assigning unique primary keys, allowing for iterative creation of data records that meet specific criteria, thereby reducing the need for extensive data identification and sanitization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single point generation process is used to move data from production environment to testing environment, then data can be extracted and made available, but the process becomes inefficient and laborious when dealing with large volumes of data requiring sanitization
Solution Approach 1:
The patent applies the copying principle by creating multiple sanitized data records from a single source data record. Instead of extracting and sanitizing numerous individual records from the production environment, the system extracts one representative record, sanitizes it by replacing NPI with fictitious data, then clones it multiple times to generate the required volume of test data. This dramatically reduces the time and resources required for data provisioning while maintaining data consistency and integration across hundreds of files and tables.
2Reliability
If sanitization algorithms are applied to remove NPI data from large volumes of records, then data security is improved, but the process becomes extremely laborious and time-consuming
Solution Approach 1:
The patent reduces sanitization complexity by performing the sanitization operation only once on a single source record rather than on thousands of records. The sanitization algorithm replaces NPI (names, addresses, phone numbers, SSNs, etc.) with fictitious data in one record, and then this sanitized record is cloned multiple times. This maintains data security through proper sanitization while reducing the operational complexity from thousands of sanitization operations to just one.
Solution Approach 2:
The patent applies preliminary action by performing the sanitization process in advance on a single representative record before cloning. By pre-sanitizing the source record and replacing all NPI elements with fictitious data, the system ensures that subsequent cloned records inherit the sanitized structure without requiring repeated sanitization operations, thereby simplifying the overall process.
3Stability of the object's composition
If multiple files and tables are pulled from production environment to maintain data associations, then integration between applications is preserved, but the volume of data requiring sanitization and processing increases significantly
Solution Approach 1:
The patent resolves this contradiction by cloning a single sanitized data record that contains references or foreign keys to related tables, rather than physically copying and sanitizing entire tables. The cloned records maintain the same relational structure and associations with other tables through consistent key values, preserving data integration consistency while processing only one source record instead of thousands of records across multiple tables.
Data Source
AI summary
Cloning of sanitized data records for subsequent provisioning in data test environments. A set of data records of a specified number that meet predefined testing criteria is requested. A single data record that meets the predefined testing criteria is pulled from a production environment. The data record is sanitized by identifying nonpublic personal information (NPI) and algorithmically replacing the NPI data with fictitious data. Subsequently, either the data record or the sanitized data record is iteratively cloned until the specified number is generated. Cloning involves algorithmically replacing the NPI data in the data record with other fictitious data or replacing the fictitious data in one of sanitized data records with other fictitious data. In addition, as a means maintaining integration between the tables and files that include the data record and the applications that use the data, the sanitization process also involves replacing a primary key value associated with the data record being sanitized with a different primary key value that is unique to other primary keys associated with other sanitized data records included in the set.