Method, computer program, and computer system for generating a set of synthetic test data (synthetic anonymized test data)

Abstract

To generate test data that closely reflects an expected usage of an application.SOLUTION: One or more processors identify private data entities present within an organic dataset; determine statistical properties of the private data entities identified within the organic dataset; create a plurality of test data templates by removing the private data entities from the organic dataset; select, from a domain knowledge base, synthetic data entities that match a data type of the removed private data entities, respectively, and align with the statistical properties of the private data entities; and generate synthetic test data by respectively inserting the synthetic data entities of the matching data type for the removed private data entities in the test data templates.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates generally to the field of application development, and more specifically to generating anonymized test data sets from organic data for testing applications under development. [Background technology]

[0002] The effectiveness and performance of an application depends on testing that is performed before it is ready for production. Application development often includes testing of application components as they are coded, and may include several layers of testing as the components are integrated into the final test version of the application. Effectiveness of application testing often depends on identifying use cases for the application that span all of the intended uses and conditions under which the application is expected to perform.

[0003] Selecting a representative data set to use to test a developed application can successfully identify expected and unexpected application behavior. Test data that closely reflects the expected usage of the application is desirable. Data elements may be referred to as data entities and may include specific data types such as numerical counts, financial statistics, transactions, and unstructured data that typically includes words and phrases, and may be intermixed with numerical data. Summary of the Invention [Problem to be solved by the invention]

[0004] It is desirable to have test data that closely reflects the expected usage of the application. [Means for solving the problem]

[0005] Several embodiments of the present invention disclose a method, computer program product, and system for generating a set of synthetic test data. The method provides that one or more processors receive an organic dataset and a domain knowledge base. The one or more processors identify private data entities present in the organic dataset. The one or more processors determine statistical characteristics of the private data entities identified in the organic dataset. The one or more processors create a plurality of test data templates by removing the private data entities from the organic dataset. The one or more processors select synthetic data entities from the domain knowledge base that respectively match data types of the removed private data entities and match statistical characteristics of the private data entities, and the one or more processors generate the synthetic test data by respectively inserting synthetic data entities of data types that match the removed private data entities in the plurality of test data templates. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a functional block diagram illustrating a distributed data processing environment in accordance with one embodiment of the present invention.

[0007] [Diagram 2] 2 is a flow chart illustrating operational steps of a model generation program operating in the distributed data processing environment of FIG. 1 according to embodiments of the present invention.

[0008] [Diagram 3] 2 is a flow chart illustrating operational steps of a data insertion program operating in the distributed data processing environment of FIG. 1 according to embodiments of the present invention.

[0009] [Figure 4]FIG. 4 illustrates a block diagram of components of a computing system including a computing device configured to operatively execute the model generation program of FIG. 2 and the data insertion program of FIG. 3 in accordance with one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Embodiments of the present invention recognize that software engineers and test engineers delivering releasable software typically require large amounts of test data that includes various corner edge cases to ensure that the delivered software meets specification requirements. Embodiments recognize that some solutions provide test data that directly uses organic data, including unedited and unfiltered data elements that can be used to identify individuals based on private identifying entities within the data. Private identifying entities include personal data and information that can be used in malicious ways and / or lead to identifying individuals, connecting individuals to private entity information. Using organic data often requires approval from multiple oversight agencies and may even require approval from individuals represented within the dataset. In some cases, using organic data may expose the group performing the software testing to privacy violations, legal repercussions, and increased risk of data leakage.

[0011]

[0004] Embodiments of the present invention recognize that to overcome the concerns and risks of using organic data, some testing organizations or providers embark on manually creating synthetic data that attempts to include the same characteristics as the initial organic data set. Such attempts are very time consuming and therefore do not represent the intended real-time response of the performance of the software under development. Data sets must be generated that account for each scenario that represents the requirements of the software, and a standard, reusable methodology that ensures adequate coverage of the test data relative to the actual data set (i.e., organic data) is typically not available.

[0012] Embodiments also recognize that semi-automatic test data generation may be attempted for part of the synthetic test data generation process. Semi-automatic generation relies on a large set of rules created by test engineers that represent the specific corner-edge use cases required by the application development specification. Rules are often considered to be of two types: data rules and constraint rules. Data rules model the initial organic dataset based on characteristics that are inherent in the dataset. Examples include a knowledge-based set of rules that specify elements or entities in a dataset that can be replaced with synthetic entities. Substitution involves using programmatic rules to transform data in one dataset to generate another dataset with synthetic entities.

[0013] Constraint rules address aspects and attributes of the data that are not inherent to the data itself, but are related to how the data is meant to be used. Constraint rules cannot be inferred from the data itself, but depend on the context of use of the data, which may require manually specifying constraints and specifying limiting the data to a subset relevant to a particular test scenario. Embodiments of the present invention recognize that a large amount of setup preparation is required to determine data rules and predictively enact constraint rules for each corner test case, and that the setup and preparation of rules is not reusable for changes to the organic data set that is the basis for the semi-automatic generation.

[0014] Embodiments of the present invention provide a method, computer program product, and computer system for automatic generation of synthetic (i.e., artificial) test data from a given source of organic test data where private entity identifiers in the data are removed and synthetic data entities, matching data types, and statistical characteristics of the original organic test data are inserted in the placeholder locations of a template version of the organic test data. Embodiments provide a completely automatic generation of synthetic data and can provide a synthetic data volume that matches a desired level by applying subtle synthetic data entity changes to the identified private data entities of the organic dataset. Embodiments do not require laborious efforts to manually create rules, but rather portions of the synthetically generated dataset can be selected to match corner edge case tests, and the generation of the synthetic dataset is not dependent on the specific organic data, which allows for easy use across multiple organic datasets.

[0015] The present invention will now be described in detail with reference to the figures. Figure 1 is a functional block diagram illustrating a distributed data processing environment, generally designated 100, in accordance with an embodiment of the present invention. Figure 1 provides only an illustration of one implementation and does not imply any limitation to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made by one of ordinary skill in the art without departing from the scope of the present invention as recited in the claims.

[0016] The distributed data processing environment 100 includes a server 110, organic test data 120, a text generation model trainer 130, and a domain knowledge base 140, all of which are interconnected via a network 150. The network 150 may be, for example, a local area network (LAN), the Internet, a wide area network (WAN) such as a virtual local area network (VLAN), or any combination that may include wired, wireless, or optical connections. In general, the network 150 may be any combination of connections and protocols that support communication and data transmission between the server 110, the organic test data 120, the text generation model trainer 130, and the domain knowledge base 140.

[0017] Organic test data 120 is a collection of data originating from the real-world data that may include undisclosed entities that are believed to identify elements of a user whose data is included in the real-world data. Organic test data 120 may include either or both structured and unstructured data. A data entity in organic test data 120 includes one or a combination of words, names, numbers, symbols, relationships, correlations, or other elements that may identify or contribute to identifying an individual or attributes of that individual whose data is included in the real-world data source of organic test data 120. Organic test data 120 is provided in a first form by the data owner and includes characteristics and content that are deemed necessary for application development test use cases. An example of organic unstructured data includes "Mary Smith is 30 years old, has two sisters and three brothers, and has lived in London for five years." An exemplary data entity of unstructured data includes " <mary> <smith> <30><2><3> <london><5> Although several embodiments of the present invention apply to either or both structured and unstructured data, the description and discussion of the concepts of the embodiments will be directed to unstructured data for convenience and clarity.

[0018] The text generation model trainer 130 receives anonymized test data templates to be used as training data for the model generator 200 to create a model that generates synthetic test data templates corresponding to the original organic test data. In one embodiment of the present invention, the text generation model trainer 130 utilizes an encoder / decoder model generation method (e.g., a sequence-to-sequence model). The text generation model trainer 130 applies a neural network (NN) function that combines word vectors with state vectors from the received anonymized test data templates. Words or placeholders from each anonymized test data template are added and encoded into the current state. Weights learned from the NN provide information on how to combine past information (NN memory vectors) and current information (new word vectors). Probability distributions are formed from multiple anonymized test data templates with supervised learning to create a trained test data generation model.

[0019] The domain knowledge base 140 includes a plurality of domain-specific terms and data elements that provide replacement words and terms used to populate the anonymized test data template to create synthetic test data. The domain knowledge base 140 can be a collection of words, terms, numbers, or similar types of phrases. In some embodiments of the invention, the domain knowledge base 140 consists of a dictionary of first and last names, and includes a list of countries, states or provinces, cities, and streets. In other embodiments, the domain knowledge base 140 has a classification structure of locations (i.e., country, state, city, street).

[0020] In yet another embodiment, the domain knowledge base 140 may include a portion of web-based content such as DBpedia, which contains organized content from information created by the Wikipedia enterprise. This structural information is available on the World Wide Web, allowing users to semantically query Wikipedia resources, including links to other related datasets. Previous releases of the DBpedia dataset included 6.0×10 6 Describe the entities, of which 5.2 × 10 6 are classified into a consistent ontology, and 1.5 × 10 6 Person, 810×10 3 Location: 135×10 3 Music album, 106×10 3 Movie, 20×10 3 Video games, 275x10 3 of organizations, 301 x 10 3 seeds, and 5×10 3 The domain knowledge base 140 may include data about numbers, relational terms, and other data types found in the organic test data 120. The domain knowledge base 140 provides a source of words, numbers, and other entities used in generating synthetic test data.

[0021] The server 110 is a computing device that hosts the model generation program 200 and the data insertion program 300 and is configured to operate in a distributed environment. The server 110 is communicatively connected to the organic test data 120, the text generation model trainer 130, and the domain knowledge base 140 via the network 150. In some embodiments, the server 110 may be a blade server, a web server, a laptop computer, a desktop computer, a standalone mobile computing device, a smartphone, a tablet computer, or other electronic device or computing system capable of receiving, transmitting, and processing data. In other embodiments, the server 110 may be a computing device that interacts with hosted applications and services and operates in a cloud computing environment. In other embodiments, the server 110 may be a netbook computer, a personal digital assistant (PDA), or other programmable electronic device capable of generating input, transmitting the input to the event logging program 300, and receiving program instructions from the event logging program 300. Alternatively, in some embodiments, server 110 may be communicatively connected to a model generation program 200 and a data insertion program 300 (not shown) operating remotely. Server 110 may include internal and external hardware components, which are illustrated in more detail in FIG.

[0022] The model generator 200 is a software application illustrated running on the server 110. The model generator 200 includes a data classification module 202, an anonymization module 204, and a statistical analysis module 206, and is communicatively coupled to the text generation model trainer 130 via the network 150. The model generator 200 determines private data entities in the organic test data 120, generates anonymized test data templates, and determines statistical properties and derived rules within the data. The model generator 200 interacts with the text generation model trainer 130 to create a synthetic test data generation module.

[0023] The data classification module 202 is a component module of the model generator 200. The data classification module 202 receives the organic test data 120, analyzes the content, and identifies non-public data entities. The data classification module 202 classifies the identified non-public data entities in the organic test data 120. In some embodiments, the non-public data entities in the organic test data 120 that should be protected are identified by using a Data Privacy Toolkit (DPT).

[0024] The anonymization module 204 is a component module of the model generator 200. The anonymization module 204 extracts private data entities of the organic test data 120 identified by the data classification module 202 and replaces the extracted private data entities with template placeholders that correspond to the type of data extracted. The anonymization module 204 performs an extract-replace operation on the entire dataset of the organic test data 120, which results in a set of test data templates that are used by the text generation model trainer 130 to generate and train a synthetic test data model.

[0025] For example, the data classification module 202 identifies the first name, last name, and age of individuals included in the organic test data 120. The anonymization module 204 extracts the first name identified by the data classification module 202 and anonymizes the extracted name.<first_name> Similarly, the anonymization module 204 extracts last names in the organic test data 120 and replaces the last names with placeholders such as<last_name> and replace each person's first and last name with the associated numeric age placeholder <age>The model generator 200 sends the collection of anonymized test data templates to the text generation model trainer 130, which is used to train the synthetic test data model.

[0026] The statistical analysis module 206 is a component module of the model generator 200. The statistical analysis module 206 receives the private data entities classified by the data classification module 202 and the anonymization module 204 and performs analysis to determine statistical characteristics of the private data entities of the organic test data 120. The statistical analysis module 206 produces statistical characteristics including distribution of the private data entity types (i.e., first name, location, age, relatives) and characteristics including correlations between the private data entity types. The statistical analysis module 206 produces analysis results that are characteristic of the dataset of entities of the organic test data 120.

[0027] Additionally, the statistical analysis module 206 provides induced constraint rules that capture correlations discovered between data entities, such as age restrictions associated with individuals who have a driver's license. The statistical analysis module 206 also provides statistical properties of the private data entities that illustrate an initial distribution of the entities, such as the number of individuals in the organic test data 120 with the first name Mary, or the relative number of individuals who are located in London, Berlin, New York, and Toronto. The statistical analysis module 206 maintains the statistical properties and correlations by mimicking the determined patterns of the organic test data 120 and applying the patterns to the synthetic test data by including the property, correlation, and rule information in the text generation model trainer 130.

[0028] The data insertion program 300 is a software application illustrated running on the server 110. The data generation program 300 generates synthetic test data including statistical properties and derived rules determined from the organic test data. The data insertion program 300 inserts data entities that are artificial with respect to the organic test data into placeholder positions of anonymized test data templates that match entity types and characteristics. The data insertion program 300 creates the synthetic test data and includes a synthetic data entity generation module 302, a synthetic data insertion module 304, and a synthetic data filtering module 306.

[0029] The synthetic data entity generation module 302 is a component module of the data insertion program 300 that receives the statistical properties of the private data entities determined by the statistical analysis module 206 and applies the properties to the domain knowledge base 140. The synthetic data entity generation module 302 generates synthetic data entities that are similar to, but not necessarily identical to, the organic test data 120, matching the data types and having the same correlations, features, and statistical properties as the private data entities in the organic test data 120.

[0030] The synthetic data insertion module 304 is a component module of the data insertion program 300. The synthetic data insertion module 302 utilizes synthetic test data templates and text generation models to replace undisclosed data entities found in the organic test data by inserting similar types of synthetic data entities at the placeholder locations of the synthetic test data templates. The synthetic data insertion module 304 inserts matching types of synthetic data specified by the placeholders of the synthetic test data templates while preserving the statistical properties, correlations, and characteristics of the original test data. As a simple example, if the original organic test data set contains a particular frequency of the first name "Mary" and the location "Paris", the synthetic data insertion module 304 maintains a similar frequency between synthetic data entities such as "Naomi" and "Berlin".

[0031] The synthetic data filtering module 306 is a component module of the data insertion program 300 that provides an option to filter the generated synthetic test data to include or remove portions of the synthetic test data based on the specific requirements of the application use case being developed. The synthetic data filtering module 306 manually applies the generated rules and constraints to the synthetic data. For example, a specific use case for testing an application may apply to a specific geography, such as North America, or a specific location, such as Geneva, Switzerland. The synthetic data filtering module 306 allows a test engineer to govern specific corner edge test areas of an application.

[0032] Embodiments of the present invention similarly apply to the structure data in which the schemes and data structures are determined and maintained in the process of generating synthetic test data, as described above.

[0033] 2 is a flow chart illustrating operational steps of a model generation program 200 operating in the distributed data processing environment 100 of FIG. 1 according to embodiments of the present invention. The model generation program 200 provides training data to a data generation module, such as a sequence-to-sequence model generator, and utilizes a model to generate synthetic data entities that match the statistical characteristics, features, and patterns of the data determined in the initial organic test data. As previously mentioned, the model generation program 200 includes a data classification module 202, an anonymization module 204, and a statistical analysis module 206. Detailed steps performed by the modules are referenced herein with respect to the model generation program 200.

[0034] The model generator 200 receives organic test data (step 210). The model generator 200 receives a data set for application testing that includes original entities that are considered private and potentially expose private or personal data that identifies or is associated with an individual. In some embodiments of the invention, the model generator 200 stores the organic test data used to generate the synthetic test data and deletes the received original test data from all memory after creating the text generation model and the synthetic test data. Embodiments of the invention recognize that consent has been given by the relevant authorities to initially receive the organic test data for the purposes of using the attributes, content types, and patterns of the original test data to generate the synthetic test data.

[0035] The model generator 200 performs classification of private data entities of the organic test data (step 220). The model generator 200 analyzes the received organic test data content and identifies private data entities contained in the organic test data. In some embodiments of the present invention, private data entities are identified by using regular expressions, dictionaries, or patterns of codes that characterize private and sensitive data, such as "@" and domain detection in emails, English name dictionaries, zip codes valid for an area, or medical terms and condition / treatment codes. The identified private data entities include content that is believed to enable the identification of an individual, where the data applies or includes personal or private information of the individual. By way of example, the model generator 200 classifies the private data entities identified in the organic test data 120 and determines the type of data, such as first name, last name, age, country location, and city location. In some embodiments, private data entities in the organic test data 120 that should be protected are identified by using a data privacy detection and management toolkit.

[0036] For example, the model generator 200 analyzes the organic test data 120 and identifies non-public data entities associated with individuals included in the organic test data 120. The model generator 200 identifies the entities and determines the type of data, such as first name, last name, relationship data, location data, and age data.

[0037] The model generator 200 replaces the private data entities of the organic test data with template placeholders (step 230). Having determined the private data entities and the types of the private entities in the organic test data, the model generator 200 extracts the private data entities from the initial organic test data and inserts placeholders for the extracted private data entities. The initial organic test data becomes a number of synthetic test data templates that maintain the initial structure but contain placeholders instead of the private data entities. The placeholders contain information indicating the type of the extracted data entities.

[0038] For example, the model generator 200 accesses the classified private data entities identified from the organic test data 120. The model generator 200 searches for each instance of the private data entity in the unstructured organic test data, such as the first name and last name, and extracts an instance of the first name (i.e., Mary) and an instance of the last name (i.e., Smith). The model generator 200 replaces the location held by the first name with a placeholder "first_name0001" that contains information about the type of data entity extracted from the location in the organic data. The information indicates the type of entity data and may provide multiple instances of the type of entity data, such as the first name "Mary" occurring 100 times.

[0039] The model generator 200 determines statistical distributions and characteristics of the private data entities in the organic test data (step 240). The model generator 200 performs an analysis of the categorized private data entities of the initial organic test data that produces statistical characteristics of the private data entities, which may include entity frequencies and correlations between entities. Additionally, the model generator 200 determines characteristics of the private data entities and patterns within the data. For example, the model generator 200 may determine that there are 827 instances of the first name "Mary" in the organic test data, of which 233 reside in London, but there are no instances of the first name "Mary" residing in Paris. The model generator 200 may determine that 421 instances of the first name "Mary" are under the age of 35, but only 374 have a driver's license.

[0040] The model generator 200 creates a data generation module by training it with anonymized test data templates (step 250). In one embodiment, for unstructured data, the model generator 200 utilizes a sequence-to-sequence neural network model and a text generation trainer application and applies the anonymized test data templates to the trainer application to generate a synthetic data generation module. In some embodiments of the invention, the text generation trainer trains a model that enables the model generator 200 to generate synthetic test data templates that correspond to the original organic test data without the undisclosed data entities.

[0041] In one embodiment of the present invention, the model generator 200 utilizes an encoder / decoder model generation method (e.g., a sequence-to-sequence model). The model generator 200 applies a neural network (NN) model that, for example, combines word embedding vectors with state vectors from the received anonymized test data templates. Words or placeholders of each anonymized test data template are added and encoded to the current state. Weights learned from the NN provide information on how to combine past information (NN memory vectors) and current information (new word vectors). Probability distributions are formed from the multiple anonymized test data templates with supervised learning to create a trained test data generation model. The text generation trainer 130 receives the anonymized test data templates and applies weights to the unstructured word components and the template placeholders such that the model can replace the placeholders by adding words to the template that make sense with the rest of the template. The model generator 200 creates a synthetic test data model that can create multiple versions of a template sentence / phrase by repeatedly applying different terms to the placeholders, recalling previous words / terms applied to the placeholders, and then applying the new terms.

[0042] The model generation program 200 provides the data generation module to the data insertion program 300 (step 260). Having generated the data generation module (for some unstructured data, text generation models), the model generation program 200 provides the data generation module access to the data insertion program 300 and terminates. In some embodiments of the invention, the data generation module is modified by submitting new or additional organic test data sets and processing them through the model generation program 200.

[0043] Figure 3 is a flow chart illustrating operational stages of a data insertion program 300 operating in the distributed data processing environment of Figure 1 according to multiple embodiments of the present invention. The data insertion program 300 accesses the data generation module created by the model generation program 200 and generates synthetic test data by insertion of placeholder synthetic data entities of a synthetic test data template.

[0044] The data insertion program 300 receives the statistical distributions and characteristics of the data entities (step 310). The data insertion program 300 receives the statistical characteristics, correlations, patterns, and features determined during analysis of the initial organic test data by the model generation program 200. The data insertion program 300 utilizes the statistical characteristics of the organic test data to determine the frequencies, correlations, and patterns to apply to the synthetic data entities, preserving the characteristics of the initial organic test data.

[0045] For example, the data insertion program 300 receives statistical characteristics, correlations, and patterns of the organic test data 120 from the analysis performed by the statistical analysis module 206 of the model generation program 200. The statistical characteristics may indicate the frequency of a private data entity, such as first name, including correlations with other private data entities, such as location and age. The data insertion program 300 retains the statistical characteristics, correlations, and patterns to apply in generating synthetic test data to maintain similarity to the organic test data 120.

[0046] The data insertion program 300 receives the contents of a domain knowledge base (step 320). The data insertion program 300 accesses and receives the contents of a domain knowledge base that includes a large number of data entities for a particular domain of data, such as personal information of an individual. In some embodiments, the data insertion program 300 accesses and receives the contents from multiple domain knowledge bases that correspond to the data entity types of the organic test data. The data insertion program 300 determines the data entity content of the domain knowledge base that matches the data types of the private data entities included in the statistical characteristics of the organic test data. In some embodiments, the domain knowledge base for the unstructured organic test data may include a dictionary of words, names, locations, may include location classifications (i.e., country, state / province, city, street) or other data sets.

[0047] The data insertion program 300 selects synthetic data entities (step 330). The data insertion program 300 applies the statistical properties, correlations, and patterns determined from the initial organic test data to the contents of the domain knowledge base to select data entities that match the data types and are similar to the private data entities of the organic test data. For example, the data insertion program 300 accesses the domain knowledge base 140 and determines a knowledge base that includes a list of first names, last names, ages, cities of location, and relationship types. In one example, the initial organic test data includes a list of first names, last names, and cities of location. The data insertion program 300 selects the list of first names, last names, and cities of location from the contents of the domain knowledge base 140 and removes the data entities of age and relationship type.

[0048] The data insertion program 300 generates the synthetic test data by inserting synthetic data entities into the respective synthetic test data templates (stage 340). The data insertion program 300 receives the respective synthetic test data templates including placeholders, each of which indicates a particular data type to be replaced. The data insertion program 300 inserts the synthetic data entities retrieved from the domain knowledge base into the respective synthetic test data templates in place of the synthetic test data template placeholders. The synthetic test data templates maintain the original format of the unstructured data. The data insertion program 300 inserts the synthetic data entities selected from the domain knowledge base and corresponding to the same data type of the original organic test data in the position of the placeholders.

[0049] For example, the data insertion program 300 receives a synthetic test data template that includes placeholders for private data entities created and extracted from the organic test data 120. The data insertion program 300 inserts synthetic data entities, such as the first name "Naomi," that correspond to a data type of "first name" selected from the domain knowledge base 140 and indicated from information associated with the placeholders. As the synthetic data entities are inserted into the synthetic test data template, the data insertion program 300 maintains statistical properties and correlations associated with the organic test data 120.

[0050] The data insertion program 300 filters the synthetic test data by applying data-induced and manually entered constraints (stage 350). The analysis of the organic test data and extracted private data entities performed by the model generation program 200 includes determining induced rules within the data itself, in addition to statistical properties and correlations. The data insertion program 300 applies the induced rules as the selection and insertion of synthetic data entities is performed, resulting in filtering of the synthetic test data entities inserted into the placeholder positions of the synthetic test data template.

[0051] For example, the derived rules may determine that the initial organic test data does not include any data associated with the first name "Mary" combined with the city location of "Paris." Similarly, test data containing information about individuals who have a driver's license will have a minimum age level, since individuals cannot obtain a driver's license until they reach legal driving age. Thus, the synthetic test data will reflect similar associations of individuals who have a driver's license and who are at or above the minimum legal driving age.

[0052] Additionally, the data insertion program 300 may receive an optional set of constraint rules to be applied to the generation of synthetic test data to provide additional filtering. For example, for a particular corner-edge use case of application testing, the synthetic test data may need to be reduced to a particular age range or subset of locations. The constraint rules are manually entered into the data insertion program 300, which applies additional constraints in generating and filtering the resulting synthetic test data, including synthetic data entities. The data insertion program 300 removes synthetic test data that does not comply with the filtering rules. As another example, electronic purchasing transaction data from using a credit card for an online purchase typically includes positive quantities. However, in the case of a refund or adjustment, the transaction may include negative quantities. The application test engineer may manually indicate a constraint to include negative transaction quantities in the synthetic test data.

[0053] The data insertion program 300 generates synthetic test data (stage 360). The data insertion program 300 completes the insertion of synthetic data entities for each synthetic test data template and applies the data derived rules and manually added constraint rules, which creates the synthetic test data. In embodiments of the present invention, the amount of synthetic test data is not limited by the amount of the initial organic test data. By varying the selection of synthetic data entities and including additional data entities from the domain knowledge base as sources, the data insertion program 300 can create a near unlimited amount of test data while maintaining the properties and correlations determined in the initial organic data set. With the synthetic test data generated, the data generation program 300 ends.

[0054] FIG. 4 illustrates a block diagram of components of a computing system including a computing device 405 configured to include or operatively connect to the components illustrated in FIG. 1 and capable of operatively executing the model generation program 200 of FIG. 2 and the data insertion program 300 of FIG. 3, respectively, in accordance with one embodiment of the present invention.

[0055] According to an exemplary embodiment of the invention, computing device 405 includes components and functionality similar to those of server 110 (FIG. 1). It should be understood that FIG. 4 is intended to illustrate only one implementation and is not intended to suggest any limitation to the environments in which different embodiments may be implemented. Many modifications to the illustrated environment may be made.

[0056] Computing device 405 includes a communications fabric 402 that provides communications between computer processor 404, memory 406, persistent storage 408, communications units 410, and input / output (I / O) interfaces 412. Communications fabric 402 may be implemented with any architecture designed to pass data and / or control information between processors (such as microprocessors, communications processors, and network processors), system memory, peripheral devices, and any other hardware components in the system. For example, communications fabric 402 may be implemented with one or more buses.

[0057] The memory 406, the cache memory 416, and the persistent storage 408 are computer-readable storage media. In this embodiment, the memory 406 includes random access memory (RAM) 414. In general, the memory 406 may include any suitable volatile or non-volatile computer-readable storage medium.

[0058] In one embodiment, the model generation program 200 and the data insertion program 300 are stored in persistent storage 408 for execution by one or more of the respective computer processors 404 via one or more of the memories 406. In this embodiment, persistent storage 408 includes a magnetic hard disk drive. Alternatively, or in addition to a magnetic hard disk drive, persistent storage 408 may include a solid state hard drive, a semiconductor storage device, a read only memory (ROM), an erasable programmable read only memory (EPROM), a flash memory, or any other computer readable storage medium capable of storing program instructions or digital information.

[0059] The media used by persistent storage 408 may also be removable. For example, a removable hard drive may be used for persistent storage 408. Other examples include optical and magnetic disks, thumb drives, and smart cards that are inserted into a drive for transfer to other computer-readable storage media that are also part of persistent storage 408.

[0060] The communications unit 410, in these examples, provides for communication with other data processing systems or devices, including resources of the distributed data processing environment 100. In these examples, the communications unit 410 includes one or more network interface cards. The communications unit 410 may provide communications through the use of either or both physical and wireless communications links. The model generation program 200 and the data insertion program 300 may be downloaded to the persistent storage 408 through the communications unit 410.

[0061] The I / O interface 412 allows for the input and output of data with other devices that may be connected to the computing system 400. For example, the I / O interface 412 may provide a connection to external devices 418, such as a keyboard, a keypad, a touch screen, or some other suitable input device or combination thereof. The external devices 418 may also include portable computer-readable storage media, such as, for example, thumb drives, portable optical or magnetic disks, and memory cards. Software and data used to implement embodiments of the invention, such as the model generation program 200 and the data insertion program 300, may be stored on such portable computer-readable storage media and loaded onto the persistent storage 408 via the I / O interface 412. The I / O interface 412 also connects to a display 420.

[0062] Display 420 provides a mechanism for displaying data to a user and may be, for example, a computer monitor.

[0063] The programs described herein are identified based on the application for which they are implemented in a particular embodiment of the invention. However, it should be understood that the names of any particular programs herein are used merely for convenience, and thus the invention should not be limited to use solely with any particular application identified and / or suggested by such names.

[0064] The present invention may be a system, method, and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions that cause a processor to perform aspects of the present invention.

[0065] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs) or flash memories, static random access memories (SRAMs), portable compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge structures in grooves with instructions recorded thereon, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not intended to be construed as being inherently a transitory signal, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or an electrical signal transmitted over a wire.

[0066] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to the respective computing / processing device or to an external computer or storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may comprise copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or combinations thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing / processing device.

[0067] The computer readable program instructions for carrying out the operations of the present invention may be either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or source or object code written in any combination of one or more programming languages, including object oriented programming languages ​​such as Smalltalk, C++, or the like, and procedural programming languages ​​such as the "C" programming language or similar. The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server, as a stand-alone software package. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), and the connection to the external computer may be made (e.g., through the Internet using an Internet Service Provider). In some embodiments, to carry out aspects of the invention, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may execute computer readable program instructions by utilizing state information of the computer readable program instructions to customize the electronic circuitry.

[0068] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0069] These computer readable program instructions may be provided to a computer processor or other programmable data processing apparatus that produces a machine, such that the instructions executing via the computer processor or other programmable data processing apparatus create means for implementing the functions / operations identified in the block or blocks of the flowcharts and / or block diagrams. These computer readable program instructions may also be stored on a computer readable storage medium that may direct a computer, programmable data processing apparatus, and / or other device to function in a particular manner, such that the computer readable storage medium having instructions stored thereon comprises an article of manufacture including instructions that implement aspects of the functions / operations identified in the block or blocks of the flowcharts and / or block diagrams.

[0070] The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device that causes a sequence of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, such that the instructions executing on the computer, other programmable apparatus, or other device implement the functions / operations identified in the flowchart and / or block diagram blocks or sections.

[0071] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or part of instructions, comprising one or more executable instructions for implementing a particular logical function. In some alternative implementations, the functions mentioned in the blocks may occur in a different order than the order mentioned in the figures. For example, two blocks shown in succession may in fact be performed as one step, or may be performed simultaneously, substantially simultaneously, partially or fully in a time-overlapping manner, and the blocks may sometimes be performed in reverse order depending on the functionality required. It will also be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a special-purpose hardware-based system that performs a particular function or operation, or executes a combination of special-purpose hardware and computer instructions.< / age> < / london> < / smith> < / mary>

Claims

1. 1. A method for generating a set of synthetic test data, comprising: receiving, by one or more processors, an organic dataset and a domain knowledge base; the one or more processors identifying non-public data entities present within the organic data set; the one or more processors determining statistical characteristics of the non-public data entities identified within the organic dataset; creating, by the one or more processors, a plurality of test data templates by removing the non-public data entities from the organic data set; selecting, by the one or more processors, synthetic data entities from the domain knowledge base that each match a data type of the removed private data entity and that match the statistical characteristics of the private data entity; generating synthetic test data by inserting, by the one or more processors, the synthetic data entities of the matching data type into the deleted private data entities in the plurality of test data templates, respectively; A method comprising:

2. generating a test data generation model using the one or more processors; training, by the one or more processors, the test data generation model based on the plurality of test data templates and the statistical characteristics of the identified non-public data entities of the organic dataset; The method of claim 1 further comprising:

3. determining, by the one or more processors, rules applicable to the synthetic test data derived by statistical analysis of the organic data set; the one or more processors filtering the synthetic test data based on the rules derived by the statistical analysis of the organic data set; removing, by the one or more processors, from the synthetic test data, synthetic test data that does not conform to the rules derived by the statistical analysis of the organic data set; The method of claim 1 or 2, further comprising:

4. the one or more processors replacing the non-public data entities to be removed from the organic data set with placeholders; the one or more processors using the placeholder to include an indicator of the data type of the private data entity to be deleted; The method of claim 1 , further comprising:

5. Determining the statistical characteristics of the non-public data entities of the organic data set includes: the one or more processors determining correlations between and patterns of the non-public data entities within the organic data set; The method of claim 1 , further comprising:

6. The method of claim 1 , wherein the synthetic test data is entered manually and filtered by constraint rules corresponding to corner use case tests.

7. The method of claim 1 , wherein the amount of synthetic test data is generated to meet a required amount based on varying the synthetic data entities selected from the domain knowledge base.

8. On the computer, receiving an organic data set and a domain knowledge base; identifying non-public data entities present within said organic data set; determining statistical characteristics of the non-public data entities identified within the organic data set; creating a plurality of test data templates by removing the unpublished data entities from the organic data set; selecting synthetic data entities from the domain knowledge base that respectively match data types of the removed private data entities and that match the statistical characteristics of the private data entities; generating synthetic test data by inserting the synthetic data entities of the matching data type into the deleted private data entities in the plurality of test data templates, respectively; 2. A computer program for generating a set of synthetic test data, the computer program causing:

9. The computer includes: A procedure for generating a test data generation model using a sequence-to-sequence model; training the test data generation model based on the plurality of test data templates and the statistical characteristics of the identified non-public data entities of the organic dataset; The computer program product of claim 8 , further comprising:

10. The computer includes: determining rules applicable to the synthetic test data derived from a statistical analysis of the organic data set; filtering the synthetic test data based on the rules derived from the statistical analysis of the organic data set; removing from the synthetic test data any synthetic test data that does not comply with the rules derived from the statistical analysis of the organic data set; 10. The computer program product according to claim 8 or 9, further comprising:

11. The computer includes: replacing the unpublished data entities to be removed from the organic data set with a placeholder; using the placeholder to include an indicator of the data type of the non-public data entity to be deleted; 11. The computer program product according to claim 8, further comprising:

12. The step of determining the statistical characteristics of the non-public data entities of the organic data set includes: determining correlations between and patterns of the non-public data entities within the organic data set; 12. The computer program of claim 8, further comprising:

13. 13. The computer program product of claim 8, wherein the synthetic test data is entered manually and filtered by constraint rules corresponding to corner use case tests.

14. 14. A computer program product as claimed in any one of claims 8 to 13, wherein the amount of synthetic test data is generated to meet a required amount based on varying the synthetic data entities selected from the domain knowledge base.

15. 1. A computer system for generating a set of synthetic test data, comprising: one or more computer processors; one or more computer readable storage media; program instructions stored on said one or more computer readable storage media; Equipped with The program instructions include: program instructions for receiving an organic data set and a domain knowledge base; program instructions for identifying non-public data entities present within the organic data set; program instructions for determining statistical characteristics of the non-public data entities identified within the organic data set; program instructions for creating a plurality of test data templates by removing the non-public data entities from the organic data set; program instructions for selecting synthetic data entities from the domain knowledge base that each match a data type of the removed private data entities and that match the statistical characteristics of the private data entities; program instructions for generating synthetic test data by respectively inserting the synthetic data entities of the matching data type into the deleted private data entities in the plurality of test data templates; A computer system comprising:

16. A test data generation model is generated using the sequence-to-sequence model; training the test data generation model based on the plurality of test data templates and the statistical characteristics of the identified private data entities of the organic dataset; 16. The computer system of claim 15, further comprising program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more processors.

17. determining rules applicable to the synthetic test data derived from a statistical analysis of the organic data set; filtering the synthetic test data based on the rules derived from the statistical analysis of the organic data set; removing from the synthetic test data any synthetic test data that does not comply with the rules derived from the statistical analysis of the organic data set; 17. A computer system according to claim 15 or 16, further comprising program instructions stored on said one or more computer readable storage media for execution by at least one of said one or more processors.

18. replacing the unpublished data entities to be removed from the organic data set with placeholders; Using the placeholder to include an indicator of the data type of the private data entity to be deleted.

18. The computer system of claim 15, further comprising program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more processors.

19. Determining the statistical characteristics of the non-public data entities of the organic data set includes: Determine correlations between the private data entities and patterns of the private data entities within the organic data set.

19. The computer system of claim 15, further comprising program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more processors.

20. 20. The computer system of claim 15, wherein the amount of synthetic test data is generated to meet a required amount based on varying the synthetic data entities selected from the domain knowledge base.

Images