Test data generation method, device and equipment
The generation of test data through data multiplication and Cartesian product operations solves the problem of time-consuming and inefficient generation of test data in the prior art, and achieves the effect of quickly generating massive test data.
Patent Information
- Application Number
- CN202510596446.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-09
- Publication Date
- 2025-08-26
AI Technical Summary
In the prior art, manual generation of test data is time-consuming and inefficient in the verification of aging of massive data processing.
Through data multiplication processing and Cartesian product calculation, massive test data are quickly generated, including multiplying the character data to unit data volume, Cartesian product calculation based on the target data volume, and generating test data based on the initial data.
It significantly shortens the time to generate massive test data, improves testing efficiency, and meets the timeliness verification needs of applications in production environments.
Smart Images
Figure CN120540976A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of testing technology, and in particular to a method, device and equipment for generating test data. Background Art
[0002] In software testing, we can simulate the data scale of the production environment in the test environment, conduct batch processing tests on massive data for application (APP) projects, and obtain the time required for the application to process data in the test environment, so as to evaluate whether it can meet the timeliness requirements in actual production.
[0003] In related technologies, testers typically manually perform repetitive tasks to generate test data to verify the timeliness of application data processing. However, this manual test data generation method is time-consuming and inefficient when verifying the timeliness of massive data processing in an application. Summary of the Invention
[0004] The present application provides a method, apparatus and device for generating test data, which are used to solve the problem in the prior art that manually generating test data is time-consuming and inefficient when verifying the processing time of massive data.
[0005] In a first aspect, the present application provides a method for generating test data, the method comprising:
[0006] Obtaining a test data construction request, wherein the test data construction request includes a unit data volume, a target data volume, and initial data;
[0007] Performing data multiplication processing on the character data to the unit data amount to obtain unit data;
[0008] According to the target data volume, performing a Cartesian product operation on the unit data to obtain expanded data;
[0009] Test data is generated according to the target data volume, the initial data, and the expanded data.
[0010] In a possible implementation, performing a Cartesian product operation on the unit data according to the target data volume to obtain expanded data includes:
[0011] If the data volume of the unit data is greater than or equal to the target data volume, determining the extended data in the unit data based on the target data volume;
[0012] If the data size of the unit data is smaller than the target data size, a Cartesian product operation is performed on the unit data to obtain temporary data, and the extended data is determined based on the unit data and the temporary data.
[0013] In a possible implementation, determining the extended data based on the unit data and the temporary data includes:
[0014] determining whether the data volume of the temporary data is greater than or equal to the target data volume;
[0015] If so, determining the extended data in the temporary data based on the target data volume;
[0016] If not, a Cartesian product operation is performed on the unit data and the temporary data until the data amount of the temporary data is greater than or equal to the target data amount, and the extended data is determined in the temporary data based on the target data amount.
[0017] In a possible implementation, generating test data according to the target data volume, the initial data, and the expanded data includes:
[0018] determining sample data in the initial data;
[0019] The test data is generated according to the sample data, the target data volume, and the expanded data.
[0020] In a possible implementation, generating the test data according to the sample data, the target data volume, and the expanded data includes:
[0021] Performing a Cartesian product operation on the sample data and the expanded data to obtain initial test data;
[0022] If the data volume of the initial test data is greater than or equal to the target data volume, generating the test data from the initial test data based on the target data volume and the sample data;
[0023] If the data volume of the initial test data is smaller than the target data volume, a Cartesian product operation is performed on the expanded data and the initial test data until the data volume of the initial test data is greater than or equal to the target data volume, and the test data is generated by the initial test data based on the target data volume and the sample data.
[0024] In a possible implementation, generating the test data based on the target data volume and the sample data using the initial test data includes:
[0025] Selecting intermediate data of the same size as the target data from the initial test data;
[0026] In the intermediate data, fields matching the sample data are selected to generate the test data.
[0027] In a possible implementation, the initial data, the unit data, the expanded data, the temporary data, the initial test data, and the test data are stored in a table format, where each row of the table is a data record and each column is a data attribute.
[0028] In a possible implementation, the target data volume is the number of data items or the number of rows in which data is stored in a table.
[0029] In a second aspect, the present application provides a device for generating test data, the device comprising:
[0030] An acquisition module, configured to acquire a test data construction request, wherein the test data construction request includes a unit data volume, a target data volume, and initial data;
[0031] A first processing module is used to perform data multiplication processing on the character data to the unit data amount to obtain unit data;
[0032] a second processing module, configured to perform a Cartesian product operation on the unit data according to the target data volume to obtain expanded data;
[0033] The third processing module is used to generate test data according to the initial data and the expanded data.
[0034] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor;
[0035] The memory stores computer-executable instructions;
[0036] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.
[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementation methods of the first aspect.
[0038] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.
[0039] The test data generation method, device and equipment provided in the embodiments of the present application include obtaining a test data construction request including a unit data amount, a target data amount and initial data, performing data multiplication processing on character data to the unit data amount to obtain unit data; then, according to the target data amount, performing a Cartesian product operation on the unit data to obtain expanded data, and finally, combining the target data amount, the initial data and the expanded data to generate test data. In the above process, through data multiplication processing and Cartesian product operation, the data scale can be quickly expanded according to the required target data amount when the initial data amount is limited, significantly shortening the time required to generate massive test data and improving test efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0041] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application;
[0042] Figure 2 A flowchart of a first embodiment of a method for generating test data provided in this application;
[0043] Figure 3 A flowchart of Example 2 of the method for generating test data provided in this application;
[0044] Figure 4 A flowchart of Example 1 of a method for generating test data provided in an embodiment of the present application;
[0045] Figure 5 A flowchart of Example 2 of a method for generating test data provided in an embodiment of the present application;
[0046] Figure 6 A flowchart of Example 3 of the test data generation method provided in an embodiment of the present application;
[0047] Figure 7 A schematic diagram of the structure of a device for generating test data provided in an embodiment of the present application;
[0048] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
[0049] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION
[0050] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
[0051] First, let’s explain the terms involved in this application:
[0052] Production environment: refers to the operating environment that is officially deployed and used by end users, and can perform batch processing of massive data according to user needs.
[0053] Test environment: A running environment specially set up for application testing, in which testers can simulate data generation and batch data processing.
[0054] Table: A table used to store data in a database. It consists of rows (records) and columns (fields). Each row represents a data record, and each column represents a field, which is used to store a specific type of data. Tables are the basic unit for organizing and storing data in a database management system, allowing users to store, query, and manage data in a structured manner.
[0055] Massive data: refers to extremely large-scale data, which usually reaches billions or even trillions of records.
[0056] Figure 1 This is a schematic diagram of the application scenario provided by the embodiment of this application. Figure 1 By constructing and processing a single piece of initial data, multiple pieces of test data can be generated on demand. Each piece of test data can maintain a standardized data structure and reasonable business logic association, making it suitable for application scenarios with massive test data, such as software testing, algorithm training, system stress testing, etc.
[0057] For example, the initial data is data 1. After data construction processing, n test data can be generated, including data 1, data 2, ..., data n. The numbers can be incremented sequentially, and field values can be generated according to preset rules, ultimately forming complete test data for software testing work that verifies the processing time of massive data.
[0058] In related technologies, testers typically generate test data manually by performing repetitive operations to verify the timeliness of application data processing. However, when it is necessary to verify the timeliness of application processing of massive amounts of data, this manual method of generating test data is often time-consuming and inefficient.
[0059] In response to the above problems, the inventors discovered in the process of studying how to quickly generate massive test data that it is possible to quickly generate massive test data in a state with less initial data through data multiplication processing and Cartesian product operation. Based on this, the inventors discovered through many experiments that after the electronic device obtains the test data construction request, the character data can be multiplied to the unit data amount to obtain unit data, and the unit data can be subjected to Cartesian product operation according to the target data amount required by the test data to obtain extended data. Then, based on the target data amount included in the test data construction request, the initial data and the extended data obtained by the Cartesian product operation, a massive amount of test data is generated, thereby improving the efficiency of data processing time verification. Based on this, the present application proposes a method for generating test data, which is used to reduce the time spent on generating test data and improve data production efficiency.
[0060] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.
[0061] Figure 2 This is a flow chart of Example 1 of the test data generation method provided in this application. Figure 2 , the method comprising:
[0062] S201: Obtain a test data construction request, where the test data construction request includes a unit data volume, a target data volume, and initial data.
[0063] The execution subject of the embodiments of the present application may be an electronic device, or a test data generation device provided in the electronic device. The test data generation device may be implemented by software, or by a combination of software and hardware. The test data generation device may be a processor in the electronic device. For ease of understanding, the following description will be based on an example in which the execution subject is an electronic device.
[0064] In this step, the electronic device may obtain a test data construction request including a unit data amount, a target data amount, and initial data.
[0065] Optionally, in the test environment of the application, the electronic device can receive manually input requests from the user interface, and the user enters relevant parameters through a graphical interface or command line. Secondly, the electronic device can read predefined requests from configuration files, which may be in JSON, XML or other formats and stored in the file system of the electronic device. In addition, the electronic device can receive requests from external services through a network interface, such as through an external application programming interface (API) call. This enables the electronic device to effectively obtain and process test data construction requests to meet different testing requirements.
[0066] The unit data volume can be measured by the number of data items or the number of rows in a table used to store data. For example, 2048 data items can be used as the unit data volume, or 2048 rows in a table can be used as the unit data volume. For ease of understanding, the unit data volume is expressed as a specific number of rows in a table.
[0067] In addition, the unit data volume can also use the byte size of the data, the number of fields, or the complexity of a specific data structure as a measurement standard, which is not limited in this application.
[0068] Similarly, the target data volume may refer to the number of data items to be generated or constructed, or the number of rows in which the test data to be generated or constructed is stored in a table.
[0069] Optionally, the initial data can be obtained from a variety of sources, including a database, an external API, a file system, or manual user input. The initial data can be stored in the form of a table, where each row of the table is a piece of data and each column is a data attribute.
[0070] For example, the electronic device may obtain a test data construction request 1, which includes a unit data volume of 2048 rows, a target data volume of 1234567890 rows, and an initial data table 1 storing initial data.
[0071] S202 , performing data multiplication processing on the character data to obtain unit data.
[0072] In this step, the character data may be multiplied until the unit data amount is reached to obtain unit data, wherein the unit data may be stored in the form of a table.
[0073] For character data, it can be any one of letters, numbers, symbols, or any combination thereof.
[0074] Optionally, you can obtain a unit table for storing unit data by following these steps:
[0075] Step 1: Create a unit table containing one field of type varchar(1) and no primary key.
[0076] Step 2: Insert a row of character data into the unit table as the basis for subsequent operations;
[0077] Step 3: Determine whether the number of rows in the unit table is equal to the unit data volume. If so, end; if not, proceed to step 4;
[0078] Step 4: Query the data in the unit table to determine whether the number of rows in the unit table after this doubling is greater than the unit data volume. If so, use the unit data volume to limit the number of rows in the query result and insert the query result back into the unit table to double the data to the unit data volume. If not, execute step 5.
[0079] Step 5: Insert the query results back into the unit table and repeat steps 3 and 4 until the number of rows in the unit table reaches the required amount of unit data.
[0080] For example, a row of character data in a unit table may be multiplied to 2048 rows of character data to obtain unit data, wherein the unit data may include 2048 rows of identical character data.
[0081] S203. Perform a Cartesian product operation on the unit data according to the target data volume to obtain expanded data.
[0082] In this step, a Cartesian product operation can be performed on the unit data based on the target data volume to generate, thereby obtaining extended data. Specifically, if the unit data volume is greater than or equal to the target data volume, the extended data is determined based on the target data volume. If the unit data volume is less than the target data volume, a Cartesian product operation is performed on the unit data to obtain temporary data, and the extended data is determined based on the unit data and the temporary data. The extended data and temporary data can be stored in a table format.
[0083] Optionally, you can obtain a large data table for storing expanded data by following these steps:
[0084] Step 1: You can create a large data table and a temporary data table with the same structure as the unit table by copying. The temporary data table is used to store temporary data, and the large data table is used to store expanded data.
[0085] Step 2: If the number of rows in the unit data is greater than or equal to the target data volume, then based on the target data volume, the expanded data is determined in the unit data and inserted into the large data table;
[0086] Step 3: If the number of rows of the unit data is less than the target data volume, perform a Cartesian product operation on the unit data to obtain temporary data, and determine whether the data volume of the temporary data is greater than or equal to the target data volume. If so, determine the expanded data in the temporary data based on the target data volume; if not, perform a Cartesian product operation on the unit data and the temporary data until the data volume of the temporary data is greater than or equal to the target data volume, and determine the expanded data in the temporary data based on the target data volume and insert it into the large data table.
[0087] For example, the unit data volume corresponding to the unit table is 2048 rows, and the target data volume is 1234567890 rows. If a large data table and a temporary data table with the same structure as the unit table are created, the first Cartesian product operation can be performed on the unit table and itself (unit table) and inserted into the temporary data table. At this time, the data in the temporary data table is 4196352 rows (including the 4194304 rows obtained by the Cartesian product operation of the unit table and itself, and the original 2048 rows of the temporary data table), which is less than the target data volume of 1234567890 rows, then the unit table can be A second Cartesian product operation is performed on the bit table with itself (the temporary data table). The result is 8591428896 rows, which is greater than 1234567890 rows. Based on the target data volume of 1234567890 rows, 1238764242 rows (including the 1234567890 rows selected based on the target data volume and the 4196352 rows obtained from the first Cartesian product operation) can be determined from the temporary data result obtained by the second Cartesian product operation and inserted into the large data table. At this time, the number of rows in the large data table is 1238764242 rows. The extended data may include 2048 rows of identical character data, 4194304 rows of identical character data, and 1234567890 rows of identical character data.
[0088] Optionally, in addition to creating a large data table and a temporary data table with the same structure as the unit table by copying, you can also only create a table structure with the same structure as the unit table, that is, the large data table and the temporary data table can copy the structure of the unit table, including column definitions, data types, indexes, default values, etc., but not including data.
[0089] S204: Generate test data based on the target data volume, initial data, and expanded data.
[0090] In this step, test data can be generated based on the target data volume, initial data, and expanded data obtained by performing a Cartesian product operation on the unit data, included in the test data construction request. In a specific implementation, sample data can be determined from the initial data, and test data can be generated based on the sample data, the target data volume, and the expanded data. The test data can be stored in a table format.
[0091] In an optional implementation, the initial data, unit data, expanded data, temporary data, initial test data, and test data are stored in a table format, where each row of the table is a data record and each column is a data attribute.
[0092] For example, initial data is stored in the initial data table, unit data is stored in the unit table, expanded data is stored in the large data table, temporary data is stored in the temporary data table, initial test data is stored in the initial test data table, and test data is stored in the test data table.
[0093] Optionally, you can obtain a test data table for storing test data by following these steps:
[0094] Step 1: Create an initial test data table and a test data table with the same table structure (excluding data) as the initial data table. The initial test data table is used to store initial test data, and the test data table is used to store test data.
[0095] Step 2: Select a sample data from the initial data table and insert it into the initial test data table;
[0096] Step 3: Query all fields in the initial test data table, perform a Cartesian product with the expanded data, and use the target data volume to limit the number of rows to obtain the initial test data;
[0097] Step 4: If the amount of initial test data is greater than or equal to the target data amount, then based on the target data amount and the sample data, generate test data from the initial test data and insert it into the test data table;
[0098] Step 5: If the amount of the initial test data is less than the target data amount, perform a Cartesian product operation on the expanded data and the initial test data until the amount of the initial test data is greater than or equal to the target data amount. Based on the target data amount and the sample data, generate test data through the initial test data and insert it into the test data table.
[0099] For example, an initial test data table and a test data table having the same table structure as the initial data table 1 are created, a sample data is randomly selected from the initial data table 1, inserted into the initial test data table, all fields in the initial test data table are queried, a Cartesian product is performed with the expanded data, and the target data volume of 1234567890 rows is used for limitation, resulting in 1234567890 rows of initial test data (including the sample data previously inserted into the initial test data table). Then, 1234567890 rows of test data can be generated from the 1234567890 rows of initial test data and inserted into the test data table, where the test data of each row can be determined by selecting a field that matches the sample data from the initial test data (i.e., the unit table for storing unit data and the big data table for storing expanded data are only used to support the process of test data generation and do not directly constitute the component or content of the test data).
[0100] In an embodiment of the present application, an electronic device can obtain a test data construction request including a unit data amount, a target data amount, and initial data, perform data multiplication processing on character data to the unit data amount to obtain unit data; perform a Cartesian product operation on the unit data according to the target data amount to obtain expanded data, and generate test data according to the target data amount, the initial data, and the expanded data. In the above process, through data multiplication and Cartesian product operations, a large amount of test data can be quickly expanded, the efficiency of test data generation can be improved, and strong support can be provided for the timeliness verification of the application, ensuring that the application performs well in the production environment.
[0101] exist Figure 2 Based on the embodiment shown below, combined with Figure 3 , the method for generating the above test data is further explained in detail.
[0102] Figure 3 This is a flow chart of Example 2 of the method for generating test data provided in this application. Figure 3 , the method may include:
[0103] S301: Obtain a test data construction request, where the test data construction request includes a unit data volume, a target data volume, and initial data.
[0104] For example, the electronic device may obtain a test data construction request 2, which includes a unit data volume of 2048 rows, a target data volume of 4,000,000 rows, and an initial data table 2 storing initial data.
[0105] S302: Perform data multiplication processing on the character data to obtain unit data.
[0106] It should be noted that the execution process of step S302 can refer to the execution process of step S202, and will not be repeated here.
[0107] S303: If the data volume of the unit data is greater than or equal to the target data volume, then based on the target data volume, extended data is determined in the unit data.
[0108] In this step, based on the data volume of the unit data being greater than or equal to the target data volume, the number of rows of the unit data may be limited by the target data volume to determine the expanded data.
[0109] For example, if the target data volume is 1024 rows and the unit data volume is 2048 rows, the unit data volume can be limited to 2048 rows by setting the target data volume to 1024 rows, and 1024 rows can be selected from the unit table of the unit data in turn and inserted into the large data table used to store the expanded data.
[0110] S304: If the data volume of the unit data is smaller than the target data volume, a Cartesian product operation is performed on the unit data to obtain temporary data, and extended data is determined based on the unit data and the temporary data.
[0111] In this step, based on the fact that the data volume of the unit data is less than the target data volume, a Cartesian product operation can be performed on the unit data to obtain temporary data, and it can be determined whether the data volume of the temporary data is greater than or equal to the target data volume; if so, based on the target data volume, the extended data is determined in the temporary data; if not, a Cartesian product operation is performed on the unit data and the temporary data until the data volume of the temporary data is greater than or equal to the target data volume, and the extended data is determined in the temporary data based on the target data volume.
[0112] For example, if the unit data volume is 2048 rows and the target data volume is 4000000 rows, a Cartesian product operation can be performed on the unit data volume (stored in the unit table) and itself (the unit table). The result is 4194304 rows, which is greater than the target data volume of 4000000 rows. Based on the target data volume of 4000000 rows, 4002048 rows (including the target data volume of 4000000 rows and the original 2048 rows of the created temporary data table) can be determined from the Cartesian product operation result and inserted into the large data table. In this case, the number of rows in the large data table is 4002048 rows. The expanded data can include 2048 rows of the same character data and 4000000 rows of the same character data.
[0113] S305: Determine sample data in the initial data.
[0114] In this step, at least one piece of sample data may be determined in the initial data table storing the initial data.
[0115] Optionally, a filtering operation may be performed in the initial data table using Structured Query Language (SQL) to randomly select a piece of data from the initial data table, or to select a specific sample (such as the maximum value, the minimum value, the first record, etc.) as sample data.
[0116] In another optional implementation, if the initial data table for storing initial data does not contain any data, the tester may insert a piece of data into the initial data table as sample data according to the test requirements.
[0117] For example, if there are 3 rows of data (ie, 3 pieces of initial data) in the initial data table 2, a filtering operation can be performed in the initial data table 2 through the SQL tool, and the first data record in the initial data table 2 is used as the sample data 1.
[0118] S306: Generate test data based on the sample data, target data volume, and expanded data.
[0119] In this step, a Cartesian product operation can be performed on the sample data and the expanded data to obtain initial test data; if the data volume of the initial test data is greater than or equal to the target data volume, test data is generated from the initial test data based on the target data volume and the sample data; if the data volume of the initial test data is less than the target data volume, a Cartesian product operation is performed on the expanded data and the initial test data until the data volume of the initial test data is greater than or equal to the target data volume, and test data is generated from the initial test data based on the target data volume and the sample data.
[0120] In an optional embodiment, based on the target data volume and the sample data, the test data is determined in the initial test data, including: selecting intermediate data of the same size as the target data volume in the initial test data; and selecting fields that match the sample data in the intermediate data to generate test data.
[0121] For example, if the target data volume is 4,000,000 rows; the sample data is 1 row (sample data 1), and the expanded data is 4,002,048 rows, then a Cartesian product operation can be performed on the sample data row 1 and the expanded data row 4,002,048 rows to obtain 4,002,048 rows of initial test data (stored in the initial test data table); since the data volume of the initial test data, 4,002,048 rows, is greater than the target data volume, 4,000,000 rows, then based on the target data volume, 4,000,000 rows, and 1 row of sample data, 4,000,000 rows of intermediate data are determined in the initial test data, and among the 4,000,000 rows of intermediate data, fields that match the sample data 1 are selected to determine 4,000,000 rows of test data (stored in the test data table).
[0122] It should be noted that with the growth of data, the increase of indexes, and the growth of log files, the unit table used to store unit data, the temporary data table used to store temporary data, and the large data table used to store extended data can be dynamically decided whether to be deleted based on the remaining size of the table space. If the table space is insufficient, the unit table, temporary data table, and large data table can be deleted. When the test data is generated for the second time, the unit table, temporary data table, and large data table can be re-established.
[0123] Similarly, the test data table used to store test data can be inserted into the initial data table according to test requirements, and the test data table can be deleted to complete the release of the test data table space.
[0124] For example, if the target data volume is 4,000,000 rows and there are 100 rows of data in the initial data table, then based on the target data volume, 3,999,900 rows of data can be selected from the test data table and inserted into the initial data table so that the number of rows of data in the initial data table is consistent with the target data volume.
[0125] In an embodiment of the present application, an electronic device can obtain a test data construction request, which may include a unit data amount, a target data amount, and initial data. Character data can then be multiplied until the unit data amount is reached to form unit data. If the unit data amount is greater than or equal to the target data amount, extended data is extracted from the unit data; if the unit data amount is less than the target data amount, a Cartesian product operation is performed on the unit data to generate temporary data, and the extended data is determined in combination with the unit data and the temporary data. Sample data is selected from the initial data, and test data is generated based on the sample data, the target data amount, and the extended data. In the above process, the multiplication process and the Cartesian product operation can quickly generate massive test data to meet the needs of large-scale testing.
[0126] Furthermore, the test data generation method provided in the embodiments of the present application can flexibly generate datasets that meet different requirements and adapt to various testing scenarios by adjusting the unit data volume and target data volume. In addition, the entire test data generation process can be automated, reducing manual intervention and improving the efficiency of test preparation.
[0127] exist Figure 2 or Figure 3 Based on the embodiment shown below, combined with Figure 4 The following example 1 further illustrates in detail the process of generating unit data by multiplying character data to reach the unit data amount in the test data generation method.
[0128] Figure 4This is a flow chart of Example 1 of the method for generating test data provided in the embodiment of this application. Figure 4 , the method comprising:
[0129] S401. Create a unit table (Unit table).
[0130] In this step, you can create a unit table named Unit, which contains one field of varchar(1) type and does not set a primary key.
[0131] A field is a column in a data table that is used to store a certain type of data. Each row (record) has a corresponding value in this field.
[0132] varchar(1) is used to represent the data type of a field. varchar represents a variable-length string. `varchar(1)` means that this field can store a string of length 1, meaning that a row can only store one character. This character can be a letter, number, symbol, etc.
[0133] A primary key is a field used to uniquely identify each row (record) in a table. Not setting a primary key means that no field or set of fields in the organization is designated to uniquely identify each row of data. This means that duplicate records are allowed in the table.
[0134] S402. Insert a row of data into the unit table.
[0135] In this step, you can insert a row of character data into the unit table based on the field type.
[0136] For example, you can insert a field of character data "A" into the unit table.
[0137] S403: Whether the number of rows in the unit table is consistent with the unit data volume.
[0138] In this step, it can be determined whether the number of rows in the unit table is consistent with the unit data volume. If so, the process ends; otherwise, step S404 is executed. Optionally, the unit data volume can be measured by the number of data items or the number of rows used to store data in the table.
[0139] For example, if the number of rows in the unit table is 1 and the unit data volume is 2048 rows, that is, the number of rows in the unit table is inconsistent with the unit data volume, then step S404 is executed.
[0140] S404. Query all data in the unit table.
[0141] For example, you can query all the data in the unit table using SQL language and find that there is one row of data in the unit table.
[0142] S405: Whether the unit table is larger than the unit data volume after multiplication.
[0143] In this step, the unit table may be multiplied once, and the number of rows of the multiplied data may be compared with the unit data volume to determine whether the function of the multiplied unit table is greater than the unit data volume. If so, execute S406; otherwise, execute S407.
[0144] For example, if the unit data volume is 2048 rows, after doubling the data of the unit table once, it can be determined that the doubled data is 3000 rows, which is larger than the unit data volume of 2048 rows, and step S406 can be executed; on the contrary, if the unit data volume is 2048 rows, after doubling the data of the unit table once, it can be determined that the doubled data is 1024 rows, which is smaller than the unit data volume of 2048 rows, and step S407 can be executed.
[0145] S406: Use unit data volume to limit the number of result rows.
[0146] In this step, you can limit the number of rows in the query result by the unit data volume.
[0147] For example, you can use a unit data size of 2048 rows to limit the number of rows in the query result to 2048 rows.
[0148] S407: Insert the queried data into the unit table to obtain unit data.
[0149] In this step, the queried data can be inserted into the unit table according to the query result until the row position of the unit table reaches 2048 rows of unit data, thereby obtaining the unit data.
[0150] For example, you can use the limit clause to limit the number of rows in the query result, such as "insert into Unit select * from Unit", until the Unit table contains 2048 rows.
[0151] The test data generation method provided in the embodiments of the present application constructs a unit table for storing individual characters and employs data multiplication to rapidly expand character-based data to a specified unit data volume, thereby generating the required unit data. By setting the unit data volume for automated multiplication, the unit table can automatically adjust its size without human intervention to match the preset data volume requirements, improving data generation efficiency and enabling testers to quickly prepare large-scale test data, thereby enhancing the overall efficiency of the testing process.
[0152] Based on any of the above embodiments, Figure 5,Through the following example 2, the process of performing Cartesian product operation on unit data ,according to the target data volume in the test data generation method, ,to obtain the expanded data is further explained in detail.
[0153] Figure 5 This is a flow chart of Example 2 of the method for generating test data provided in the embodiment of this application. Figure 5 , the method comprising:
[0154] S501. Create a large data table and a temporary data table based on the unit table.
[0155] In this step, a temporary data table for storing temporary data and a large data table for storing expanded data can be created by copying.
[0156] For example, you can create a large data table (n_big table) and a temporary data table (n_tmp table) based on the Unit table.
[0157] S502: Obtain target data volume.
[0158] For example, the target data volume that can be obtained is 1234567890 rows.
[0159] S503: Whether the number of rows in the unit table is greater than or equal to the target data volume.
[0160] In this step, the number of rows in the unit table may be compared with the target data volume. If the number of rows in the unit table is greater than or equal to the target data volume, step S504 is executed; if not, step S505 is executed.
[0161] For example, step S505 may be executed based on the number of rows in the Unit table being 2048, which is smaller than the target data volume of 234567890 rows.
[0162] S504: Query all data in the unit table, limit the number of rows using the target data volume, and insert the data into the large data table.
[0163] In this step, based on the number of rows in the unit table being greater than or equal to the target data volume, the target data volume may be used to limit the number of rows, and the large data table may be inserted to obtain expanded data.
[0164] For example, if the target data volume is 2000 rows, then since the number of rows in the unit table (2048) is larger than the target data volume (2000 rows), we can limit the number of rows using the target data volume of 2048 and insert the data into the n_big table to obtain 2000 rows of expanded data. It should be understood that during the test data construction process, the number of rows in the target data volume is usually much larger than the number of rows in the unit table.
[0165] S505: Perform a Cartesian product operation on the unit table and itself, and insert the result into a temporary data table.
[0166] For example, you can perform a Cartesian product operation on 2048 rows of data in the Unit table and 2048 rows of data in the Unit table, and insert them into the n_tmp table. The resulting number of data rows in the n_tmp table is 4196352 (including the 4194304 rows obtained by performing a Cartesian product operation on the Unit table and itself, and the original 2048 rows in the created n_tmp table).
[0167] S506: Whether the number of rows in the temporary data table is greater than or equal to the target data volume.
[0168] In this step, it is determined whether the number of rows in the temporary data table is greater than or equal to the target data volume. If so, S507 is executed; if not, S505 is executed.
[0169] S507: Query all data in the temporary data table, limit the number of rows using the target data volume, and insert the data into the large data table.
[0170] For example, it can be determined that the number of rows in the temporary data table is 4196352 rows, which is less than the target data volume of 1234567890 rows. Then, step S505 and step S506 are executed once, and a Cartesian product operation is performed on the Unit table and the n_tmp table. According to the operation result, 8591428896 rows are obtained, which is greater than the target data volume of 1234567890 rows. Based on the target data volume of 1234567890 rows, 1238764242 rows (including the 1234567890 rows selected by the target data volume and the 4196352 rows obtained by the first Cartesian product operation) are determined in the temporary data result obtained by the second Cartesian product operation and inserted into the n_big table. At this time, the number of rows of expanded data in the n_big table is 1238764242 rows.
[0171] S508: Delete the temporary data table.
[0172] In an optional implementation, the temporary data table may occupy a large amount of storage space during the data generation process. Deleting the temporary data table can free up database table space and reduce storage resource consumption.
[0173] The test data generation method provided in this application dynamically compares the unit table data volume with the target data volume requirement, intelligently selects direct interception or Cartesian product operation to achieve exponential data growth, and then selects the required amount of data to insert into the large data table after the conditions are met. This method can provide strong support for generating massive test data in a short period of time, thereby significantly improving the efficiency and scalability of test data preparation.
[0174] Based on any of the above embodiments, Figure 6,Through the following example three, the process of generating test data according to the target data volume, initial data and expanded data in the test data generation method is further explained in detail.
[0175] Figure 6 This is a flow chart of Example 3 of the test data generation method provided in the embodiment of this application. Figure 6 , the method comprising:
[0176] S601: Create an initial test data table and a test data table according to the initial data table.
[0177] In this step, an initial test data table and a test data table having the same table structure as the initial data table may be created, wherein the initial data table is used to store the initial data in the test data construction request.
[0178] For example, if the initial data table is table A, an initial test data table (table B) and a test data table (table C) having the same table structure as table A are created.
[0179] S602: Whether the number of rows in the initial data table is greater than or equal to 1.
[0180] In this step, if there is at least one piece of data in the initial data table, step S603 is executed; if not, step S604 is executed.
[0181] S603: Insert a piece of data into the initial data table.
[0182] In this step, after determining that no data exists in the initial data table, a piece of data for testing (also one row of data) can be inserted into the initial data table according to the test requirements of the application.
[0183] S604: Filter a piece of data from the initial data table and insert it into the initial test data table.
[0184] For example, you can randomly select a piece of data from table A and insert it into table B as sample data.
[0185] S605: Query all fields of the initial test data table, perform Cartesian product operation with the large data table, use the target data volume to limit the number of rows, and insert the data into the initial test data table.
[0186] S606: Initially test whether the number of rows in the data table is greater than or equal to the target data volume.
[0187] In this step, if the number of rows in the initial test data table is greater than or equal to the target data volume, step S607 may be executed; otherwise, step S605 may be continued.
[0188] For example, all fields in table B can be queried and a Cartesian product operation can be performed with table n_big (number of rows is 1238764242). The number of rows can be limited by the target data volume of 1234567890 rows, and inserted into the initial test data table. The initial test data in the initial test data table is obtained to have 1234567890 rows, and step S607 is executed.
[0189] S607: Generate test data using the initial test data and store it in a test data table.
[0190] For example, the test data for each row in table C can be determined by selecting fields from the initial test data that match the sample data.
[0191] In an alternative embodiment, based on the target data volume, the initial data table may already contain some data, and the remaining required data volume (i.e., the target data volume minus the partial data volume in the initial data table) may be selected from the test data table and inserted into the initial data table. Subsequently, the test data table may be deleted to free up the space it occupies.
[0192] The test data generation method example provided in the embodiment of the present application creates an initial test data table and a test data table that are consistent with the structure of the initial data table, and uses Cartesian product operations and target data volume limits to quickly expand the data volume and insert the results into the initial test data table. When the target data volume is reached, the generated test data is stored in the test data table. In the above process, the efficiency and flexibility of test data generation are ensured, and test efficiency is improved. In addition, the required amount of data is extracted from the test data table and inserted into the initial data table, and the test data table is deleted at the same time, which can free up table space and improve storage efficiency.
[0193] Figure 7 This is a schematic diagram of the structure of the test data generation device provided in the embodiment of the present application. Figure 7 , a test data generating device 10, comprising:
[0194] An acquisition module 11 is configured to acquire a test data construction request, wherein the test data construction request includes a unit data volume, a target data volume, and initial data;
[0195] The first processing module 12 is used to perform data multiplication processing on the character data to the unit data amount to obtain unit data;
[0196] A second processing module 13 is configured to perform a Cartesian product operation on the unit data according to the target data volume to obtain expanded data;
[0197] The third processing module 14 is configured to generate test data according to the initial data and the expanded data.
[0198] The test data generation device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar and will not be repeated here.
[0199] In a possible implementation manner, the second processing module 13 is specifically configured to:
[0200] If the data volume of the unit data is greater than or equal to the target data volume, determining the extended data in the unit data based on the target data volume;
[0201] If the data size of the unit data is smaller than the target data size, a Cartesian product operation is performed on the unit data to obtain temporary data, and the extended data is determined based on the unit data and the temporary data.
[0202] In a possible implementation manner, the second processing module 13 is specifically configured to:
[0203] determining whether the data volume of the temporary data is greater than or equal to the target data volume;
[0204] If so, determining the extended data in the temporary data based on the target data volume;
[0205] If not, a Cartesian product operation is performed on the unit data and the temporary data until the data amount of the temporary data is greater than or equal to the target data amount, and the extended data is determined in the temporary data based on the target data amount.
[0206] In a possible implementation, the third processing module 14 is specifically configured to:
[0207] determining sample data in the initial data;
[0208] The test data is generated according to the sample data, the target data volume, and the expanded data.
[0209] In a possible implementation, the third processing module 14 is specifically configured to:
[0210] Performing a Cartesian product operation on the sample data and the expanded data to obtain initial test data;
[0211] If the data volume of the initial test data is greater than or equal to the target data volume, generating the test data from the initial test data based on the target data volume and the sample data;
[0212] If the data volume of the initial test data is smaller than the target data volume, a Cartesian product operation is performed on the expanded data and the initial test data until the data volume of the initial test data is greater than or equal to the target data volume, and the test data is generated by the initial test data based on the target data volume and the sample data.
[0213] In a possible implementation, the third processing module 14 is specifically configured to:
[0214] Selecting intermediate data of the same size as the target data from the initial test data;
[0215] In the intermediate data, fields matching the sample data are selected to generate the test data.
[0216] In a possible implementation, the initial data, the unit data, the expanded data, the temporary data, the initial test data, and the test data are stored in a table format, where each row of the table is a data record and each column is a data attribute.
[0217] In a possible implementation, the target data volume is the number of data items or the number of rows in which data is stored in a table.
[0218] The test data generation device provided in the embodiment of the present application can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar and will not be repeated here.
[0219] Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 8 The electronic device 20 includes: at least one processor 21 and a memory 22. Optionally, the electronic device 20 further includes a communication component 23. The processor 21, the memory 22 and the communication component 23 are connected via a bus 24.
[0220] During the specific implementation process, at least one processor 21 executes the computer-executable instructions stored in the memory 22, so that the at least one processor 21 performs the above method.
[0221] The specific implementation process of the processor 21 can be found in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here in this embodiment.
[0222] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASICs), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly executed by a hardware processor or by a combination of hardware and software modules in the processor.
[0223] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk memory.
[0224] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be classified into address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.
[0225] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.
[0226] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.
[0227] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.
[0228] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.
[0229] The division of units is merely a logical functional division; actual implementations may employ alternative divisions, such as combining or integrating multiple units or components into another system, or omitting or disabling certain features. Furthermore, any direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units, either through an interface, electrical, mechanical, or other means.
[0230] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
[0231] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0232] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.
[0233] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.
[0234] Finally, it should be noted that those skilled in the art will readily identify other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art not disclosed herein. The present invention is not limited to the precise structure described above and illustrated in the accompanying drawings, and various modifications and variations may be made without departing from the scope thereof. The scope of the present invention is limited solely by the appended claims.
Claims
1. A method for generating test data, characterized in that: The method comprises: Obtaining a test data construction request, wherein the test data construction request includes a unit data volume, a target data volume, and initial data; Performing data multiplication processing on the character data to the unit data amount to obtain unit data; According to the target data volume, performing a Cartesian product operation on the unit data to obtain expanded data; Test data is generated according to the target data volume, the initial data, and the expanded data.
2. The method according to claim 1, characterized in that The step of performing a Cartesian product operation on the unit data according to the target data volume to obtain expanded data includes: If the data volume of the unit data is greater than or equal to the target data volume, determining the extended data in the unit data based on the target data volume; If the data size of the unit data is smaller than the target data size, a Cartesian product operation is performed on the unit data to obtain temporary data, and the extended data is determined based on the unit data and the temporary data.
3. The method according to claim 2, characterized in that The determining the extended data based on the unit data and the temporary data includes: determining whether the data volume of the temporary data is greater than or equal to the target data volume; If so, determining the extended data in the temporary data based on the target data volume; If not, a Cartesian product operation is performed on the unit data and the temporary data until the data amount of the temporary data is greater than or equal to the target data amount, and the extended data is determined in the temporary data based on the target data amount.
4. The method according to claim 1, wherein Generating test data according to the target data volume, the initial data, and the expanded data includes: determining sample data in the initial data; The test data is generated according to the sample data, the target data volume, and the expanded data.
5. The method according to claim 4, characterized in that Generating the test data according to the sample data, the target data volume, and the expanded data includes: Performing a Cartesian product operation on the sample data and the expanded data to obtain initial test data; If the data volume of the initial test data is greater than or equal to the target data volume, generating the test data from the initial test data based on the target data volume and the sample data; If the data volume of the initial test data is smaller than the target data volume, a Cartesian product operation is performed on the expanded data and the initial test data until the data volume of the initial test data is greater than or equal to the target data volume, and the test data is generated by the initial test data based on the target data volume and the sample data.
6. The method according to claim 5, characterized in that Generating the test data by using the initial test data based on the target data volume and the sample data includes: Selecting intermediate data of the same size as the target data from the initial test data; In the intermediate data, fields matching the sample data are selected to generate the test data.
7. The method according to claim 3, characterized in that The initial data, the unit data, the expanded data, the temporary data, the initial test data, and the test data are stored in a table format, where each row of the table is a data record and each column is a data attribute.
8. The method according to claim 7, characterized in that The target data volume is the number of data items or the number of rows in which data is stored in a table.
9. A device for generating test data, characterized in that: The device comprises: An acquisition module, configured to acquire a test data construction request, wherein the test data construction request includes a unit data volume, a target data volume, and initial data; A first processing module is used to perform data multiplication processing on the character data to the unit data amount to obtain unit data; a second processing module, configured to perform a Cartesian product operation on the unit data according to the target data volume to obtain expanded data; The third processing module is used to generate test data according to the initial data and the expanded data.
10. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 8.