A method and device for obtaining test data

Abstract

The present invention provides a method and device for obtaining test data, which can be used in the financial field or other technical fields. The method comprises: extracting data based on the data extraction rules of each first data table and the backup file of the production data to obtain the first-stage data extraction result; wherein the data extraction rule of each first data table is preset; obtaining the second-stage data extraction result based on the data extraction rules of each second data table, the first-stage data extraction result and the backup file; wherein the data extraction rule of each second data table is preset; obtaining the test data based on the first-stage data extraction result and the second-stage data extraction result. The device is used to execute the above method. The method and device for obtaining test data provided by the embodiment of the present invention reduce the amount of test data.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a method and device for obtaining test data. Background Art

[0002] During the process of software product development and online testing, software developers often need to use production data generated in the production environment to test the software products in a test environment.

[0003] At present, the scale of application databases is getting larger and larger, and the amount of production data stored is also increasing. Production data cannot usually be used directly to test software products, and needs to go through processes such as data desensitization, transfer, and recovery. As the amount of production data increases, the processing time for production data in order to obtain test data is also getting longer and longer, and the probability of errors in the process of processing production data is also increasing accordingly. In addition, for the testing of software products, multiple test environments are usually configured. Under the same volume of production data, a large number of storage devices are required, resulting in high testing costs.

[0004] Therefore, how to propose a method for obtaining test data, which can extract part of the data from the production data for testing the test environment and reduce the amount of test data used, has become an important issue that needs to be solved in this field. Summary of the invention

[0005] In view of the problems in the prior art, an embodiment of the present invention provides a method and device for obtaining test data, which can at least partially solve the problems in the prior art.

[0006] In one aspect, the present invention provides a method for obtaining test data, comprising:

[0007] Extract data based on the data extraction rules of each first data table and the backup file of the production data to obtain the first-stage data extraction result; wherein the data extraction rule of each first data table is preset;

[0008] Based on the data extraction rules of each second data table, the data extraction results of the first stage and the backup file, obtaining the data extraction results of the second stage; wherein the data extraction rules of each second data table are preset;

[0009] Based on the first-stage data extraction results and the second-stage data extraction results, test data is obtained.

[0010] In another aspect, the present invention provides a device for obtaining test data, comprising:

[0011] A first extraction module, used to perform first-stage data extraction based on data extraction rules of each first data table and a backup file of production data to obtain first-stage data extraction results; wherein the data extraction rules of each first data table are preset;

[0012] A second extraction module, used to obtain the second-stage data extraction result based on the data extraction rules of each second data table, the first-stage data extraction result and the backup file; wherein the data extraction rule of each second data table is preset;

[0013] The acquisition module is used to obtain test data based on the first-stage data extraction results and the second-stage data extraction results.

[0014] On the other hand, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for obtaining test data described in any of the above embodiments when executing the program.

[0015] In yet another aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for obtaining test data described in any of the above embodiments.

[0016] The test data acquisition method and device provided in the embodiment of the present invention can perform data extraction based on the data extraction rules of each first data table and the backup files of the production data to obtain the first-stage data extraction results, and then obtain the second-stage data extraction results based on the data extraction rules of each second data table, the first-stage data extraction results and the backup files, and then obtain test data based on the first-stage data extraction results and the second-stage data extraction results, thereby reducing the amount of test data while meeting the test data required by the test environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0018] Figure 1 It is a flowchart of a method for obtaining test data provided by an embodiment of the present invention.

[0019] Figure 2 It is a flowchart of a method for obtaining test data provided by another embodiment of the present invention.

[0020] Figure 3 It is a schematic diagram of the structure of a device for obtaining test data provided by an embodiment of the present invention.

[0021] Figure 4 It is a schematic diagram of the structure of a device for obtaining test data provided by another embodiment of the present invention.

[0022] Figure 5 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention. It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other arbitrarily.

[0024] In order to facilitate the understanding of the technical solution provided by the present application, the relevant contents of the technical solution of the present application are first explained below. By analyzing the application data, it is found that the distribution and use of bank data have the following characteristics: (1) The data has obvious partition characteristics. Due to historical reasons, most business type data can be divided according to area code and branch number. Except for special business, the data characteristics between regions and branches are similar; (2) The test data access is relatively concentrated. The test usually selects data from regions with a full range of business types for testing, such as Beijing, Zhejiang, etc. Such regions can cover the test range by selecting fewer account numbers, card numbers, etc.; (4) Data access popularity is highly correlated with new businesses. For example, when a new business is launched in a certain area, the frequency of data use in the area increases significantly. Based on the current problem and the above characteristics, the method for obtaining test data provided by the embodiment of the present invention is proposed, which can support an automatic data extraction scheme that flexibly configures data extraction rules according to area code, branch number, customer information number, medium number, card number, etc.

[0025] The following takes the server as an example to illustrate the specific implementation process of the method for obtaining test data provided by the embodiment of the present invention. It can be understood that the execution subject of the method for obtaining test data provided by the embodiment of the present invention includes but is not limited to the server.

[0026] Figure 1 FIG. 1 is a flow chart of a method for obtaining test data provided by an embodiment of the present invention. Figure 1 As shown, the method for obtaining test data provided by the embodiment of the present invention includes:

[0027] S101, extracting data based on the data extraction rules of each first data table and the backup file of the production data to obtain the first-stage data extraction result; wherein the data extraction rule of each first data table is preset;

[0028] Specifically, in the embodiment of the present invention, data processing is performed with data tables as dimensions. Production data is stored in a database in the form of data tables, and data backup is performed regularly to generate backup files of production data. The server can extract data from the backup files of the production data based on the data extraction rules of each first data table to obtain each first data table, and each first data table constitutes the first stage data extraction result. The data extraction rules of each first data table are preset.

[0029] For example, the first data table is an extraction table, and accordingly, the data extraction rule of the first data table is an extraction table data extraction rule, and a filter field is set in the extraction table data extraction rule. The server will query the corresponding data according to the filter field and store it in the corresponding extraction table. Among them, the filter field can be an area code, a customer information number, an organization number, a branch number, etc., which is set according to actual needs and is not limited in the embodiment of the present invention.

[0030] For example, the first data table is a full data table, and accordingly, the data extraction rule of the first data table is a full data table extraction rule, and the full data table extraction rule sets a table name. The server will copy the data table corresponding to the table name as the full data table according to the table name. The table name can be a parameter table, a product information table, etc., and is set according to actual needs, and the embodiment of the present invention does not limit it.

[0031] For example, the first data table is an empty table. Accordingly, the data extraction rule of the first data table is an empty table extraction rule. The empty table extraction rule sets the table name and limits the fields and indexes of the data table corresponding to the table name. The server will query the corresponding data table according to the table name, and copy the fields and indexes of the corresponding data table to the corresponding empty table. The obtained empty table only includes fields and indexes, and no data corresponding to the fields. For log tables, historical information tables, etc. in production data, usually only fields and indexes need to be extracted.

[0032] S102, obtaining a second-stage data extraction result based on the data extraction rule of each second data table, the first-stage data extraction result and the backup file; wherein the data extraction rule of each second data table is preset;

[0033] Specifically, after obtaining the first-stage data extraction result, the server may extract data from the first-stage data extraction result and / or the backup file based on the data extraction rule of each second data table to obtain each second data table, and each second data table constitutes the second-stage data extraction result. The data extraction rule of each second data table is preset.

[0034] For example, the second data table can be obtained by extracting data from a first extraction table and the backup file; the second data table can be obtained by extracting data from two different first extraction tables; the second data table can be obtained by extracting data from a first extraction table and a full data table; the second data table can be obtained by extracting data from a first extraction table and a full data table; the second data table can be obtained by extracting data from a second extraction table and the backup file; the second data table can be obtained by extracting data from a second extraction table and the first extraction table; the second data table can be obtained by extracting data from a second extraction table and the full data table. Among them, the second extraction table is a second data table.

[0035] S103: Obtain test data based on the first-stage data extraction results and the second-stage data extraction results.

[0036] Specifically, not all first data tables included in the first-stage data extraction results are used for testing, and there are first data tables used for data extraction of second data tables. Not all second data tables included in the second-stage data extraction results are used for testing, and there are second data tables used for data extraction of second data tables. The server may remove first data tables not used for testing from the first-stage data extraction results, and remove second data tables not used for testing from the second-stage data extraction results, and use the remaining first data tables and second data tables as test data.

[0037] The method for obtaining test data provided by the embodiment of the present invention can extract data based on the data extraction rules of each first data table and the backup file of the production data to obtain the first-stage data extraction result, and then obtain the second-stage data extraction result based on the data extraction rules of each second data table, the first-stage data extraction result and the backup file, and then obtain test data based on the first-stage data extraction result and the second-stage data extraction result, thereby reducing the amount of test data while meeting the test data required by the test environment. In addition, in the process of obtaining test data, it is not necessary to process all production data, which improves the efficiency of obtaining test data, reduces the demand for storage devices, and reduces the test cost.

[0038] On the basis of the above embodiments, further, the data extraction rule of the first data table includes a table extraction rule, a full data table extraction rule or an empty table extraction rule.

[0039] Specifically, the first data table can be an extracted table, a full data table or an empty table. An extracted table refers to a data table that directly extracts data from the backup file to obtain data. A full data table refers to a data table that directly copies the data of the original data table. An empty table refers to a data table that only obtains fields and indexes without obtaining the data corresponding to the fields.

[0040] The first data table is an extraction table, then the data extraction rule of the first data table is an extraction table data extraction rule, the extraction table data extraction rule sets a filter field, and the data corresponding to the filter field is extracted from the backup file according to the filter field and stored in an extraction table. Among them, the filter field can be one field or two or more fields. The filter field is set according to actual needs, and the embodiment of the present invention is not limited. An extraction table that is not used as test data is used to extract data from the second data table and can be called an intermediate table. The first data table is a full data table, then the data extraction rule of the first data table is a full data table extraction rule, the full data table extraction rule sets the table name, and the data table corresponding to the table name is copied according to the table name as the full data table. Among them, the table name is set according to actual needs, and the embodiment of the present invention is not limited.

[0041] If the first data table is an empty table, the data extraction rule of the first data table is an empty table extraction rule. The empty table extraction rule sets the table name and limits the fields and indexes of the data table corresponding to the table name. According to the table name, the corresponding data table is queried, and the fields and indexes of the corresponding data table are copied to the corresponding empty table. The obtained empty table only includes fields and indexes, and has no data corresponding to the fields.

[0042] For example, a data extraction rule for a table extraction includes a filtering field of area code; a data extraction rule for a table extraction includes a filtering field of area code and customer information number; a data extraction rule for a table extraction includes a filtering field of area code and branch number; a data extraction rule for a table extraction includes a filtering field of area code and account number; a data extraction rule for a table extraction includes a filtering field of branch number; a data extraction rule for a table extraction includes a filtering field of area code or branch number.

[0043] On the basis of the above embodiments, further, the data extraction rules of the second data table include:

[0044] Obtaining a target table and an associated table corresponding to the second data table;

[0045] If it is determined that the data corresponding to the preset fields in the target table and the associated table are the same, the same data are extracted into the second data table.

[0046] Specifically, the second data table is obtained by extracting the data in the target table and the associated table corresponding to the second data. The target table can be obtained from the data extraction result of the first stage or the backup file, and is set according to actual needs, which is not limited in the embodiment of the present invention. The associated table can be obtained from the data extraction result of the first stage or the second data table that has been obtained. The server obtains the target table and the associated table corresponding to the second data table, and then obtains the corresponding data from the target table according to the preset field, and queries the corresponding data from the associated table according to the preset field, and then compares the data obtained from the target table with the data obtained from the associated table. If the data is the same, the same data will be copied to the second data table. If the data is not the same, the data will not be copied to the data table.

[0047] For example, 100 data corresponding to the preset field are obtained from the target table of the second data table, and the 100 data are arranged in sequence according to the order in the target table; 100 data corresponding to the preset field are obtained from the association table of the second data table, and the 100 data are arranged in sequence according to the order in the association table. The server compares whether the i-th data obtained from the target table is the same as the i-th data obtained from the association table. If they are the same, the data is copied to the second data table; if they are not the same, the data is not copied. Wherein, i is a natural number, and i is less than or equal to 100.

[0048] On the basis of the above embodiments, further, the first-stage data extraction result includes a full data table and an extracted table; accordingly, the data extraction rule based on each second data table, the first-stage data extraction result and the backup file, obtaining the second-stage data extraction result includes:

[0049] A target table corresponding to the second data table is obtained from the full data table, the extracted table or the backup file, and an associated table corresponding to the second data table is obtained from the full data table or the extracted table.

[0050] Specifically, the data extraction result of the first stage includes a full data table and an extracted table, then the server can obtain a data table from the full data table, the extracted table or the backup file as the target table corresponding to the second data table. The server can obtain a data table from the full data table or the extracted table as the association table corresponding to the second data table. Among them, the extracted table that is not used as test data can also be called an intermediate table, which exists to obtain the second data table.

[0051] Figure 2 FIG. 1 is a flow chart of a method for obtaining test data provided by the second embodiment of the present invention. Figure 2As shown, on the basis of the above embodiments, further, the data extraction rules of each second data table, the first stage data extraction results and the backup file are used to obtain the second stage data extraction results, including:

[0052] S201, constructing an extraction sequence corresponding to the association table according to the extraction dependency relationship of the association table corresponding to the second data table; wherein the extraction dependency relationship is preset and includes at least one other second data table;

[0053] Specifically, the association table corresponding to the second data table may be a full data table, an extracted table, or other second data tables. When the association table is other second data tables, it is necessary to set the extraction dependency of the association table, that is, which data tables are required to be extracted in sequence to obtain the association table. The server may construct the extraction order corresponding to the association table based on the extraction dependency of the association table corresponding to the second data table, that is, which data table is extracted first, which data table is extracted next, and so on, to finally obtain the association table. Among them, other second data tables refer to data tables different from the second data tables corresponding to the association table.

[0054] For example, the extraction dependency of the association table is to first obtain the full data table A, then obtain the second data table B based on the full data table A, and finally obtain the association table C based on the second data table B. Then the extraction order corresponding to constructing the association table C is the full data table A, the second data table B, and the association table C.

[0055] For example, the extraction dependency of the association table is to first obtain an extraction table a, then obtain a second data table b based on the extraction table a, then obtain a second data table c based on the second data table b, and finally obtain an association table d based on the second data table c.

[0056] S202: Extract data in sequence according to the extraction order to obtain a correlation table corresponding to the second data table.

[0057] Specifically, after obtaining the extraction sequence corresponding to the association table, the server extracts data in sequence according to the extraction sequence, that is, extracts each data table required to obtain the association table in sequence, and finally obtains the association table corresponding to the second data table.

[0058] For example, if the extraction order corresponding to association table C is full data table A, second data table B, and association table C, the server will obtain full data table A from the first-stage data extraction result, and then obtain second data table B according to the data extraction rule of second data table B and full data table A, and then obtain association table C according to the data extraction rule of association table C and second data table B. When obtaining second data table B according to the data extraction rule of second data table B and full data table A, full data table A is used as the association table corresponding to second data table B. When obtaining association table C according to the data extraction rule of association table C and second data table B, second data table B is used as the association table corresponding to association table C.

[0059] On the basis of the above embodiments, further, the method for obtaining test data provided by the embodiment of the present invention also includes:

[0060] The data extraction rules of each first data table and the second-stage data extraction rules of each second data table are checked for accuracy.

[0061] Specifically, before performing the first-stage data extraction based on the data extraction rules of each first data table and the backup file of the production data and obtaining the first-stage data extraction result, the server can import the data extraction rules of each first data table and the second-stage data extraction rules of each second data table into the database, and then perform accuracy verification on the data extraction rules of the first data table and the second-stage data extraction rules of each second data table. It can be checked through the database data dictionary information whether the fields set in each data extraction rule exist. If they do not exist, it means that the field setting is wrong, and a field error can be prompted; it can be checked whether each first data table and each second data table are configured with data extraction rules. If the data extraction rules are not configured, it means that there is a lack of data extraction rules, and a prompt can be prompted that the data extraction rules are missing; it can be checked based on the data tables required by the test data whether each first data table and each second data table cover the data tables required by the measurement data. If they are not covered, it means that there is a missing data table, and a prompt can be prompted that the data table is missing.

[0062] On the basis of the above embodiments, further, the method for obtaining test data provided by the embodiment of the present invention also includes:

[0063] An integrity check is performed on the backup file.

[0064] Specifically, due to the large amount of production data, there are situations where the backup dates of different data tables or partitions of data tables span days. Therefore, the server will perform an integrity check on the backup files to check whether backup files within a continuous time range exist. If a backup file of a certain backup date is missing, it means that the backup file of the production data is incomplete and the backup file needs to be supplemented.

[0065] For example, production data is backed up regularly every day. If the backup file of the obtained production data lacks the backup data of a certain day, it means that the backup file is missing.

[0066] The first step is to import data extraction rules. The data extraction rules of each first data table and the data extraction rules of each second data table are imported into the database to form a data extraction configuration table.

[0067] Step 2: Check the data extraction configuration table. Check the data extraction configuration table to see if there are any field errors, missing data extraction rules, or missing first or second data tables.

[0068] Step 3: Determine whether the check is passed. If the extracted data table passes the check, proceed to step 4. If the extracted data table does not pass the check, correct the existing problems, such as changing the wrong fields, supplementing the data extraction rules, supplementing the missing first data table and / or second data table, etc., and then repeat the first step.

[0069] Step 4: Check the backup file. Check whether the backup file of the production data is complete. If it is complete, proceed to step 5. If it is incomplete, supplement the missing data and check again.

[0070] Step 5: Generate the first type of job tasks. Generate job tasks for each first data table, including a job task for extracting a table, that is, extracting data from the backup file based on a data extraction rule to obtain a table; a job task for an empty table, that is, extracting data from the backup file based on an empty table extraction rule to obtain an empty table; a job task for a full data table, that is, extracting data from the backup file based on a full data table extraction rule to obtain a full data table.

[0071] Step 6. Generate the second type of job tasks. Generate a job task for each second data table. The job task of the second data table is to obtain the target table and the associated table corresponding to the second data table; if it is determined that the data corresponding to the preset fields in the target table and the associated table are the same, the same data is extracted into the second data table. In order to obtain the associated table corresponding to the second data table, a job task for the associated table can be generated as a subtask of the job task for the second data table, that is, according to the extraction dependency relationship of the associated table corresponding to the second data table, an extraction order corresponding to the associated table is constructed; wherein the extraction dependency relationship is preset, including at least one other second data table, and data is extracted in sequence according to the extraction order to obtain the associated table corresponding to the second data table.

[0072] Step 7: Store test data. Obtain test data from each of the first data table and the second data table and store them in the database.

[0073] When generating the first type of job tasks and the second type of job tasks, the job priority can be configured for each job task. A higher priority can be set for the job tasks of the intermediate table in the extracted table, and a higher priority can be set for the job tasks of the associated table. For the job tasks with higher priority, priority will be given to processing and priority allocation of resources. If the extracted data table has multiple partitions during data extraction, data extraction can be performed in parallel according to the partitions to improve the efficiency of data extraction.

[0074] Among them, when performing the second type of task, you can use the IBM OPTIM product, which can directly associate the backup file with the online table query. Using this technology can save the intermediate import steps and effectively improve data processing efficiency.

[0075] The method for obtaining test data provided by the embodiment of the present invention analyzes data distribution and usage characteristics, describes extraction rules in the form of a configuration table, and divides the test data into different data tables for classification processing, thereby greatly reducing the size of the test data while meeting the use of test data in the test environment. At the same time, based on the configuration table, database, and data recovery software, the operation and process generation are generated in an automated manner, greatly improving the efficiency of data slimming work.

[0076] (1) The present invention uses the data distribution characteristics and the screening fields such as area code and network point number as data extraction conditions to effectively retain the data characteristics and reduce the impact on software product testing. Combined with the actual business distribution, key business areas are selected first, and the online and batch test ranges can be covered with a small amount of data.

[0077] (2) The present invention describes the extraction rules by configuring tables, which is easy to understand and use, and can directly convert and generate extraction conditions based on the rules. In addition, the table extraction relationship described by the rules facilitates the automatic generation of the table recovery process; the use of the intermediate table also meets the needs of special usage scenarios.

[0078] (3) The present invention can utilize offline extraction technology to avoid repeated data import, complete data extraction implementation in one go, reduce processing links, and improve data extraction efficiency.

[0079] Figure 3 FIG. 1 is a schematic diagram of a structure of a device for obtaining test data provided by an embodiment of the present invention. Figure 3 As shown, the apparatus for obtaining test data provided by the embodiment of the present invention includes a first extraction module 301, a second extraction module 302 and an acquisition module 303, wherein:

[0080] The first extraction module 301 is used to perform the first-stage data extraction based on the data extraction rules of each first data table and the backup file of the production data, and obtain the first-stage data extraction results; wherein the data extraction rules of each first data table are preset; the second extraction module 302 is used to obtain the second-stage data extraction results based on the data extraction rules of each second data table, the first-stage data extraction results and the backup file; wherein the data extraction rules of each second data table are preset; the acquisition module 303 is used to obtain test data based on the first-stage data extraction results and the second-stage data extraction results.

[0081] Specifically, in the embodiment of the present invention, data processing is performed based on the data table as a dimension. Production data is stored in the database in the form of a data table, and data backup is performed regularly to generate a backup file of the production data. The first extraction module 301 can extract data from the backup file of the production data based on the data extraction rule of each first data table to obtain each first data table, and each first data table constitutes the first stage data extraction result. Among them, the data extraction rule of each first data table is preset.

[0082] After obtaining the first-stage data extraction result, the second extraction module 302 can extract data from the first-stage data extraction result and / or the backup file based on the data extraction rule of each second data table to obtain each second data table, and each second data table constitutes the second-stage data extraction result. The data extraction rule of each second data table is preset.

[0083] Not all of the first data tables included in the first-stage data extraction results are used for testing, and there are first data tables used for data extraction of second data tables. Not all of the second data tables included in the second-stage data extraction results are used for testing, and there are second data tables used for extraction of second data tables. The acquisition module 303 can remove the first data tables not used for testing from the first-stage data extraction results, and remove the second data tables not used for testing from the second-stage data extraction results, and use the remaining first data tables and second data tables as test data.

[0084] The test data acquisition device provided by the embodiment of the present invention can extract data based on the data extraction rules of each first data table and the backup file of the production data to obtain the first-stage data extraction result, and then obtain the second-stage data extraction result based on the data extraction rules of each second data table, the first-stage data extraction result and the backup file, and then obtain the test data based on the first-stage data extraction result and the second-stage data extraction result, thereby reducing the amount of test data while meeting the test data required by the test environment. In addition, in the process of obtaining test data, it is not necessary to process all production data, which improves the efficiency of obtaining test data, reduces the demand for storage devices, and reduces the test cost.

[0085] On the basis of the above embodiments, further, the data extraction rule of the first data table includes a table extraction rule, a full data table extraction rule or an empty table extraction rule.

[0086] On the basis of the above embodiments, further, the data extraction rules of the second data table include:

[0087] Obtaining a target table and an associated table corresponding to the second data table;

[0088] If it is determined that the data corresponding to the preset fields in the target table and the associated table are the same, the same data are extracted into the second data table.

[0089] On the basis of the above embodiments, further, the first stage data extraction result includes a full data table and an extraction table; accordingly, the second extraction module 302 is specifically used for:

[0090] A target table corresponding to the second data table is obtained from the full data table, the extracted table or the backup file, and an associated table corresponding to the second data table is obtained from the full data table or the extracted table.

[0091] Figure 4 is a schematic diagram of the structure of a device for obtaining test data provided by another embodiment of the present invention. Figure 4As shown, based on the above embodiments, further, the second extraction module 302 includes a construction unit 3021 and an extraction unit 3022, wherein:

[0092] The construction unit 3021 is used to construct an extraction order corresponding to the association table according to the extraction dependency of the association table corresponding to the second data table; wherein the extraction dependency includes at least one other second data table; the extraction unit 3022 extracts data in sequence according to the extraction order to obtain the association table corresponding to the second data table.

[0093] On the basis of the above embodiments, further, the apparatus for obtaining test data provided by the embodiment of the present invention further includes a verification module:

[0094] The verification module is used to verify the accuracy of the data extraction rules of each first data table and the second-stage data extraction rules of each second data table.

[0095] On the basis of the above embodiments, further, the apparatus for obtaining test data provided by the embodiment of the present invention further includes a checking module:

[0096] The checking module is used to perform integrity check on the backup file.

[0097] The embodiments of the apparatus provided in the embodiments of the present invention can be specifically used to execute the processing flow of the above-mentioned method embodiments, and its functions are not described in detail here, and reference can be made to the detailed description of the above-mentioned method embodiments.

[0098] It should be noted that the method and device for obtaining test data provided in the embodiments of the present invention can be used in the financial field, and can also be used in any technical field other than the financial field. The embodiments of the present invention do not limit the application field of the method and device for obtaining test data.

[0099] Figure 5 is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention, such as Figure 5As shown, the electronic device may include: a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 communicate with each other through the communication bus 504. The processor 501 may call the logic instructions in the memory 503 to execute the following method: extract data based on the data extraction rules of each first data table and the backup file of the production data to obtain the first-stage data extraction result; wherein the data extraction rule of each first data table is preset; obtain the second-stage data extraction result based on the data extraction rules of each second data table, the first-stage data extraction result and the backup file; wherein the data extraction rule of each second data table is preset; obtain test data based on the first-stage data extraction result and the second-stage data extraction result.

[0100] In addition, the logic instructions in the above-mentioned memory 503 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0101] The present embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the methods provided by the above-mentioned method embodiments, for example, including: extracting data based on data extraction rules of each first data table and a backup file of production data to obtain a first-stage data extraction result; wherein the data extraction rule of each first data table is preset; obtaining a second-stage data extraction result based on data extraction rules of each second data table, the first-stage data extraction result and the backup file; wherein the data extraction rule of each second data table is preset; and obtaining test data based on the first-stage data extraction result and the second-stage data extraction result.

[0102] The present embodiment provides a computer-readable storage medium, which stores a computer program, and the computer program enables the computer to execute the methods provided by the above-mentioned method embodiments, for example, including: extracting data based on the data extraction rules of each first data table and the backup file of the production data to obtain the first-stage data extraction result; wherein the data extraction rule of each first data table is preset; obtaining the second-stage data extraction result based on the data extraction rules of each second data table, the first-stage data extraction result and the backup file; wherein the data extraction rule of each second data table is preset; obtaining test data based on the first-stage data extraction result and the second-stage data extraction result.

[0103] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0104] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0105] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0107] In the description of this specification, the description with reference to the terms "one embodiment", "a specific embodiment", "some embodiments", "for example", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0108] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for obtaining test data, characterized in that: include: Extract data based on the data extraction rules of each first data table and the backup file of the production data to obtain the first-stage data extraction result; wherein the data extraction rule of each first data table is preset; Based on the data extraction rules of each second data table, the data extraction results of the first stage and the backup file, obtaining the data extraction results of the second stage; wherein the data extraction rules of each second data table are preset; Based on the data extraction results of the first stage and the data extraction results of the second stage, obtaining test data; Wherein, each first data table includes an empty table, the data extraction rule of the first data table includes an empty table extraction rule, and the empty table extraction rule includes extracting fields and indexes of the data table corresponding to the table name; accordingly, extracting data based on the data extraction rules of each first data table and the backup file of the production data to obtain the first-stage data extraction result includes: According to the table name of the empty table, query the corresponding data table, and copy the fields and indexes of the corresponding data table to the corresponding empty table. The obtained empty table only includes fields and indexes. The data extraction rules of the second data table include: Obtaining a target table and an associated table corresponding to the second data table; If it is determined that the data corresponding to the preset fields in the target table and the associated table are the same, extracting the same data into the second data table; Among them, obtaining the association table corresponding to the second data table includes: constructing an extraction order corresponding to the association table according to the extraction dependency relationship of the association table corresponding to the second data table; wherein the extraction dependency relationship is preset and includes at least one other second data table; extracting data in sequence according to the extraction order to obtain the association table corresponding to the second data table.

2. The method according to claim 1, characterized in that The data extraction rule of the first data table further includes a table extraction rule or a full data table extraction rule.

3. The method according to claim 1, characterized in that The first-stage data extraction result includes a full data table and an extracted table; accordingly, the second-stage data extraction result is obtained based on the data extraction rules of each second data table, the first-stage data extraction result and the backup file, including: A target table corresponding to the second data table is obtained from the full data table, the extracted table or the backup file, and an associated table corresponding to the second data table is obtained from the full data table or the extracted table.

4. The method according to claim 1, characterized in that: The step of obtaining the second-stage data extraction result based on the data extraction rules of each second data table, the first-stage data extraction result and the backup file comprises: According to the extraction dependency relationship of the association table corresponding to the second data table, construct an extraction order corresponding to the association table; wherein the extraction dependency relationship is preset and includes at least one other second data table; Data is extracted in sequence according to the extraction order to obtain an associated table corresponding to the second data table.

5. The method according to claim 1, characterized in that Also includes: The data extraction rules of each first data table and the second-stage data extraction rules of each second data table are checked for accuracy.

6. The method according to any one of claims 1 to 5, characterized in that: Also includes: An integrity check is performed on the backup file.

7. A device for obtaining test data, characterized in that: include: A first extraction module, used to perform first-stage data extraction based on data extraction rules of each first data table and a backup file of production data to obtain first-stage data extraction results; wherein the data extraction rules of each first data table are preset; A second extraction module, used to obtain the second-stage data extraction result based on the data extraction rules of each second data table, the first-stage data extraction result and the backup file; wherein the data extraction rule of each second data table is preset; An acquisition module, used for acquiring test data based on the data extraction result of the first stage and the data extraction result of the second stage; Each first data table includes an empty table, the data extraction rule of the first data table includes an empty table extraction rule, and the empty table extraction rule includes extracting fields and indexes of the data table corresponding to the table name; accordingly, the first extraction module is specifically used to query the corresponding data table according to the table name of the empty table, and copy the fields and indexes of the corresponding data table to the corresponding empty table, and the obtained empty table only includes fields and indexes; The data extraction rule of the second data table includes: obtaining a target table and an associated table corresponding to the second data table; if it is determined that the data corresponding to the preset fields in the target table and the associated table are the same, extracting the same data into the second data table; Among them, obtaining the association table corresponding to the second data table includes: constructing an extraction order corresponding to the association table according to the extraction dependency relationship of the association table corresponding to the second data table; wherein the extraction dependency relationship is preset and includes at least one other second data table; extracting data in sequence according to the extraction order to obtain the association table corresponding to the second data table.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images