Data testing method, device, electronic equipment and storage medium

By sampling the data table, the amount of data is reduced and the processing efficiency is improved, the problem of low data testing efficiency in the existing technology is solved, and more efficient and accurate data testing is achieved.

CN113868141BActive Publication Date: 2025-02-28BEIJING DAJIA INTERNET INFORMATION TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202111167840.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-29
Publication Date
2025-02-28
Estimated Expiration
2041-09-29

AI Technical Summary

Technical Problem

In the prior art, data testing efficiency is low, mainly due to the large amount of data, which takes a long time.

Method used

By obtaining the data table to be tested, sampling is performed to obtain the sampling table, and replacing the data table to be tested with the sampling table, the test process is performed to obtain the sampling test results, and finally determining the test results of the data table based on the sampling test results.

Benefits of technology

The data volume is reduced through sampling, the efficiency of data processing is improved, and the comprehensiveness and accuracy of data testing is improved through the integration of multiple sampling test results, thereby improving the efficiency of data testing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113868141B_ABST
    Figure CN113868141B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a data testing method, device, electronic device and storage medium. The method includes: obtaining a data table to be tested, the data table is a collection of data information required in a test process; performing sampling processing on the data information in the data table to obtain at least one sampling table, the sampling table is used to store the data information sampled from the data table; replacing the data table to be tested with at least one sampling table, and performing a test process to obtain a sampling test result corresponding to at least one sampling table; and determining the test result of the data table according to at least one sampling test result. The method realizes sampling the data table to obtain a sampling table with less data information, thereby reducing the amount of data in the data testing process; at the same time, the sampling table is used to perform the test process, and the test result of the data table is determined according to multiple sampling test results, which also improves the comprehensiveness and accuracy of data testing on the data table, and further improves the efficiency of data testing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a data testing method, device, electronic equipment and storage medium. Background Art

[0002] With the rapid development of Internet technology, it is necessary to use existing data for testing when developing corresponding functions; after the data test is passed, the function can be put online for application.

[0003] In the related art, the test environment needs to execute data test requests in sequence, and the amount of data in the data test is usually large, resulting in a long time to complete each data test request; therefore, the existing data test efficiency is still relatively low. Summary of the invention

[0004] The present disclosure provides a data testing method, device, electronic device and storage medium to at least solve the problem of low data testing efficiency in related technologies. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided a data testing method, comprising:

[0006] Obtain a data table to be tested, wherein the data table is a collection of data information required to be called in the test process;

[0007] Performing sampling processing on the data information in the data table to obtain at least one sampling table, wherein the sampling table is used to store the sampled data information in the data table;

[0008] Replacing the data table to be tested with the at least one sampling table, and executing the test process to obtain a sampling test result corresponding to the at least one sampling table;

[0009] A test result of the data table is determined according to the at least one sampling test result.

[0010] In an exemplary embodiment, obtaining a data table to be tested includes:

[0011] In response to a test request including a task to be tested, obtaining a data table lineage relationship of the task to be tested, wherein the data table lineage relationship is used to characterize an upstream and downstream relationship between various data tables;

[0012] According to the data table lineage relationship, the data table to be tested corresponding to the task to be tested is obtained.

[0013] In an exemplary embodiment, acquiring the data table to be tested corresponding to the task to be tested according to the data table lineage relationship includes:

[0014] Constructing a data table network according to the data table blood relationship, and determining a target network branch in the data table network;

[0015] At least one data table is selected from each of the target network branches as the data table to be tested corresponding to the task to be tested.

[0016] In an exemplary embodiment, after obtaining the data table to be tested corresponding to the task to be tested, the method further includes:

[0017] Obtaining metadata information of each of the data tables to be tested;

[0018] According to the metadata information, the identifier, attribute and attribute value of the data information in each of the data tables to be tested are obtained.

[0019] In an exemplary embodiment, the sampling parameters include at least a time range, a sampling quantity, and a sampling method;

[0020] The performing sampling processing on the data information in the data table to obtain at least one sampling table includes:

[0021] Filtering data information that meets the time range from the data table;

[0022] According to the sampling method, data information of the sampling quantity is extracted from the identifier of the screened data information as data information of the sampling table.

[0023] In an exemplary embodiment, the sampling method is stratified sampling;

[0024] The method of extracting the data information of the sampling quantity from the identifier of the filtered data information according to the sampling method as the data information of the sampling table includes:

[0025] According to the attributes and the attribute values, the filtered data information is divided into multiple levels;

[0026] According to the identifier, a preset sampling quantity of data information is extracted from each level respectively, wherein the preset sampling quantity of data information is the data information of the sampling table.

[0027] In an exemplary embodiment, after performing sampling processing on the data information in the data table to obtain at least one sampling table, the method further includes:

[0028] Obtaining a first statistical result of the data table and a second statistical result of the at least one sampling table;

[0029] Comparing the first statistical result with the second statistical result to obtain a comparison result;

[0030] The matching degree between the data table and the at least one sampling table is determined according to the comparison result; the matching degree is used to determine the sampling table with the closest performance to the data table from the at least one sampling table.

[0031] In an exemplary embodiment, executing the test process to obtain a sampling test result corresponding to the at least one sampling table includes:

[0032] Obtaining a preset evaluation indicator corresponding to the test process, evaluating the sampling table according to the preset evaluation indicator, and obtaining an indicator evaluation result matching the preset evaluation indicator;

[0033] An indicator evaluation report is generated according to the indicator evaluation result, and the indicator evaluation report is used to determine the data status of the data table.

[0034] According to a second aspect of an embodiment of the present disclosure, there is provided a data testing device, comprising:

[0035] A data table acquisition unit is configured to acquire a data table to be tested, wherein the data table is a collection of data information required in the test process;

[0036] A sampling table acquisition unit is configured to perform sampling processing on the data information in the data table to obtain at least one sampling table, wherein the sampling table is used to store the sampled data information in the data table;

[0037] A data testing unit, configured to replace the data table to be tested with the at least one sampling table, and execute the test process to obtain a sampling test result corresponding to the at least one sampling table;

[0038] The result acquisition unit is configured to determine the test result of the data table according to the at least one sampling test result.

[0039] In an exemplary embodiment, the data table acquisition unit is further configured to execute in response to a test request containing a task to be tested, obtain the data table lineage relationship of the task to be tested, and the data table lineage relationship is used to characterize the upstream and downstream relationship between each data table; based on the data table lineage relationship, obtain the data table to be tested corresponding to the task to be tested.

[0040] In an exemplary embodiment, the data table acquisition unit is further configured to construct a data table network based on the data table blood relationship, determine the target network branch in the data table network; and select at least one data table from each of the target network branches as the data table to be tested corresponding to the task to be tested.

[0041] In an exemplary embodiment, the data table acquisition unit is further configured to acquire metadata information of each of the data tables to be tested; and acquire identification, attributes and attribute values ​​of data information in each of the data tables to be tested according to the metadata information.

[0042] In an exemplary embodiment, the sampling parameters include at least a time range, a sampling quantity, and a sampling method;

[0043] The sampling table acquisition unit is further configured to filter out data information that meets the time range from the data table; and extract the data information of the sampling quantity from the identifier of the filtered data information according to the sampling method as the data information of the sampling table.

[0044] In an exemplary embodiment, the sampling method is stratified sampling;

[0045] The sampling table acquisition unit is further configured to divide the filtered data information into multiple levels according to the attributes and the attribute values; and extract a preset sampling quantity of data information from each level according to the identifier, wherein the preset sampling quantity of data information is the data information of the sampling table.

[0046] In an exemplary embodiment, the sampling table acquisition unit is further configured to execute acquisition of a first statistical result of the data table and a second statistical result of the at least one sampling table; compare the first statistical result with the second statistical result to obtain a comparison result; determine a degree of match between the data table and the at least one sampling table based on the comparison result; the degree of match is used to determine, from at least one sampling table, a sampling table whose performance is closest to the data table.

[0047] In an exemplary embodiment, the data testing unit is further configured to execute and obtain preset evaluation indicators corresponding to the test process, evaluate the sampling table according to the preset evaluation indicators, obtain indicator evaluation results that match the preset evaluation indicators, and generate an indicator evaluation report according to the indicator evaluation results; the indicator evaluation report is used to determine the data status of the data table.

[0048] According to a third aspect of an embodiment of the present disclosure, a server is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the data testing method as described in any one of the first aspects.

[0049] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of a server, the server is enabled to execute the data testing method as described in any one of the first aspects.

[0050] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, wherein the computer program product includes instructions, and when the instructions are executed by a processor of a server, the server is enabled to execute the data testing method as described in any one of the first aspects.

[0051] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0052] By acquiring a data table to be tested, the data table is a collection of data information required in the test process; then performing sampling processing on the data information in the data table to obtain at least one sampling table, the sampling table is used to store the data information sampled from the data table; then replacing the data table to be tested with at least one sampling table, and executing the test process to obtain the sampling test results corresponding to the at least one sampling table; finally, according to the at least one sampling test result, determining the test result of the data table. It is achieved that at least one sampling table with less data information is obtained by sampling the data table, thereby reducing the amount of data in the data testing process and improving the data processing efficiency; at the same time, at least one sampling table is used to execute the test process, and the test result of the data table is determined according to multiple sampling test results, which also improves the comprehensiveness and accuracy of the data test on the data table, and further improves the efficiency of data testing.

[0053] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0055] Figure 1 The diagram is an application environment diagram of a data testing method according to an exemplary embodiment.

[0056] Figure 2 The figure is a flow chart of a data testing method according to an exemplary embodiment.

[0057] Figure 3 The present invention is a flowchart of a method for obtaining a data table to be tested corresponding to a task to be tested according to a data table lineage relationship according to an exemplary embodiment.

[0058] Figure 4The figure is a schematic diagram showing blood relationship according to an exemplary embodiment.

[0059] Figure 5 The present invention is a flowchart of a method for obtaining identification, attributes and attribute values ​​of data information in each data table to be tested according to metadata information according to an exemplary embodiment.

[0060] Figure 6 The present invention is a flowchart showing a schematic diagram of obtaining identification, attributes and attribute values ​​of data information through metadata information according to an exemplary embodiment.

[0061] Figure 7 The present invention is a flowchart of a method for performing sampling processing on data information in a data table to obtain at least one sampling table according to an exemplary embodiment.

[0062] Figure 8 The present invention is a flowchart of a method for executing a test process to obtain a sampling test result corresponding to at least one sampling table according to an exemplary embodiment.

[0063] Fig. 9 is a schematic diagram showing a test process executed on sampling tables of different numbers according to another exemplary embodiment.

[0064] Fig.10 The invention is a flowchart of a method for obtaining at least one sampling table according to an exemplary embodiment.

[0065] Fig.11 is a flow chart of yet another data testing method according to an exemplary embodiment.

[0066] Fig.12 The invention is a block diagram of a data testing device according to an exemplary embodiment.

[0067] Fig.13 The figure is a diagram showing the internal structure of a server according to an exemplary embodiment. DETAILED DESCRIPTION

[0068] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0069] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0070] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties; correspondingly, this disclosure also provides a corresponding user authorization entrance for users to choose to authorize or refuse.

[0071] The data testing method provided by the present disclosure can be applied to Figure 1 In the application environment shown. Among them, the terminal 110 interacts with the server 120 through the network; the server 120 obtains the data table to be tested from the terminal 110, and the data table is a collection of data information required in the test process; the server 120 performs sampling processing on the data information in the data table to obtain at least one sampling table, and the sampling table is used to store the data information sampled from the data table; the server 120 replaces the data table to be tested with at least one sampling table, and executes the test process to obtain the sampling test result corresponding to the at least one sampling table; the server 120 determines the test result of the data table according to at least one sampling test result; the server 120 sends the test result of the data table back to the terminal 110 through the network. Among them, the terminal 110 can be but not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the server 120 can be implemented by an independent server or a server cluster composed of multiple servers.

[0072] Figure 2 is a flow chart of a data testing method according to an exemplary embodiment. Figure 2 As shown, the data testing method is used as Figure 1 The server 120 in the application environment shown includes the following steps:

[0073] In step S210, a data table to be tested is obtained, where the data table is a collection of data information required in the test process.

[0074] The data table to be tested is a data table associated with a task to be tested or a code to be tested; the data table contains multiple pieces of data information. For example, when the task to be tested Q is executed, data tables Q1, Q2, and Q3 need to be called. Then the data tables Q1, Q2, and Q3 obtained from the thousands of tables in the data warehouse are the data tables to be tested associated with the task to be tested or the code to be tested; the data information in the data tables Q1, Q2, and Q3 is the data information required in the test process and can be called.

[0075] The code to be tested can be a database statement, such as a SQL (Structured Query Language) statement, which can implement functions such as adding, deleting, querying, and modifying data tables. The test process refers to the execution logic corresponding to the task to be tested or the code to be tested. For example, when the code to be tested is executed, steps A, B, and C are performed respectively, and then steps A, B, and C constitute the test process.

[0076] Specifically, the server identifies the type, quantity, time, storage location and other information of the data table required by the task to be tested and the code to be tested according to the trigger of the task to be tested and the code to be tested, and obtains the qualified data table from the data table storage medium such as the data warehouse according to the aforementioned information as the data table to be tested. It should be noted that the acquisition process of the data table to be tested is not limited to obtaining from the medium such as the data warehouse, but can also be in the form of the terminal directly sending the test table that needs to be tested to the server for the test process.

[0077] In step S220, a sampling process is performed on the data information in the data table to obtain at least one sampling table, and the sampling table is used to store the sampled data information in the data table.

[0078] Among them, the sampling table is a table file with the same format as the data table after sampling the data information in the data table. The quantity and sorting of the data information in the sampling table may vary according to the sampling parameters. One or more sampling tables can be obtained after sampling from a data table. The quantity of data information contained in different sampling tables may also vary according to the sampling parameters.

[0079] The sampling process is controlled by various types of sampling parameters, such as the number of sampling tables, time range, number of samples, sampling methods, sampling methods, etc. Sampling methods can be divided into sampling with replacement and sampling without replacement, and sampling methods can include random sampling, stratified sampling, etc. The same data table can be sampled multiple times according to different sampling parameters to obtain multiple sampling tables.

[0080] The sampling parameters may be carried by the task to be tested or the code to be tested, or may be set in advance or by using default settings.

[0081] Specifically, the server generates an empty table with the same format according to the data table to be tested in advance; then the server obtains the data tables to be tested one by one or simultaneously, and obtains corresponding data information from each data table according to pre-set sampling parameters; the extracted data information is put into the empty table, and a sampling table with the same format as the data table is obtained.

[0082] In step S230, the data table to be tested is replaced with at least one sampling table, and the test process is executed to obtain a sampling test result corresponding to the at least one sampling table.

[0083] Specifically, the amount of data information in the sampling table is less than that in the data table, so replacing the data table to be tested with the sampling table of the same format can improve the execution speed of the task to be tested and the code to be tested, and speed up the test process; and because the sampling table has the same format as the data table, the replacement process does not require unnecessary debugging, and the relationship mapped to the data table can be directly modified to the sampling table. Furthermore, since multiple sampling tables can be obtained after sampling the data table, after executing the same test process on multiple sampling tables, the test results of the data table can be more comprehensively and accurately reflected. For example, if only one sampling table is selected to execute the test process, a sampling test result A1 is obtained; if multiple sampling tables are selected to execute the test process separately, for example, three sampling tables are selected to execute the test process, three sampling test results A2, A3, and A4 are obtained respectively, and the sampling results correspond to the sampling tables that execute the test process. Since the sampling table is obtained based on sampling of the data table to be tested, the sampling test results of each sampling table can reflect the test results of the data table. If the sampling test results are close to each other, it may mean that the data table also has characteristics that match the sampling test results; if there are obvious differences between the sampling test results, it may mean that the data in the data table is not balanced or there are other problems, which lead to large differences between the sampling tables; that is, the data characteristics of the data table can be reflected by the sampling test results.

[0084] In step S240, a test result of the data table is determined according to at least one sample test result.

[0085] Among them, the sampling test result refers to the result obtained after executing the test process on at least one sampling table; the test result refers to the test result obtained after integrating at least one sampling test result, and its effect is equivalent to the result obtained after testing the data table.

[0086] In the above data testing method, at least one sampling table with less data information is obtained by sampling the data table, thereby reducing the amount of data in the data testing process and improving data processing efficiency; at the same time, at least one sampling table is used to execute the test process, and the test results of the data table are determined based on multiple sampling test results, which also improves the comprehensiveness and accuracy of data testing on the data table and further improves the efficiency of data testing.

[0087] In an exemplary embodiment, if Figure 3 As shown, in the above step S210, obtaining the data table to be tested includes the following steps:

[0088] In step S310, in response to a test request including a task to be tested, a data table lineage relationship of the task to be tested is obtained, where the data table lineage relationship is used to characterize an upstream and downstream relationship between various data tables;

[0089] Among them, the test request is the request information carrying the task to be tested sent by the terminal or other devices to the server; the task to be tested refers to the instruction information that enables the server to clearly identify the corresponding test process, the data table to be called and other information to establish the corresponding test process.

[0090] Among them, table lineage is the logic that abstractly describes the relationship between data table fields in storage media such as data warehouses; Figure 4 The figure shows a schematic diagram of table lineage relationship, in which the data table on the same branch line as data table A is the upstream data table or downstream data table of data table A. Therefore, the upstream and downstream relationship between each data table can be obtained through the table lineage relationship, so as to locate and obtain the data table in the data warehouse.

[0091] In step S320, the data table to be tested corresponding to the task to be tested is obtained according to the data table lineage relationship.

[0092] Specifically, the server obtains the data table to be tested from the data warehouse according to the upstream and downstream relationships based on the blood relationship of the data tables, and obtains the data table to be tested.

[0093] In the technical solution provided by the embodiment of the present disclosure, the data table to be tested can be accurately located in the data warehouse through the table lineage relationship. At the same time, the table lineage relationship can ensure that the selected data tables to be tested are not in an upstream and downstream relationship and have certain differences. Therefore, the data table to be tested selected according to the table lineage relationship can better reflect the characteristics of the data table, and the test results obtained by executing the test process will be more accurate.

[0094] In an exemplary embodiment, in the above step S320, based on the data table lineage relationship, the data table to be tested corresponding to the task to be tested is obtained, and the following steps are also included: constructing a data table network based on the data table lineage relationship, and determining the target network branch in the data table network; selecting at least one data table from each target network branch as the data table to be tested corresponding to the task to be tested.

[0095] Specifically, the table lineage relationship represents the connection relationship between each data table, and therefore, based on the table lineage relationship, a data table network that is interconnected between data tables can be constructed; at the same time, the table lineage relationship also records the data tables that have a lineage relationship with each other, so it can be guaranteed that the data tables that have a lineage relationship exist on the same branch of the data table network. Therefore, according to the task to be tested, the specific parameters of the required data table to be tested can be determined, and at least one network branch in the data table network is first determined based on the parameters, and then the specific data table is located from the at least one network branch according to the time range, version, update time and other parameters specified by the task to be tested, and at least one data table to be tested is obtained, and so on, until all the data tables to be tested corresponding to the task to be tested are obtained.

[0096] For example, the table of blood relationship records that data tables a1, a2, and a3 are blood-related, b1, b2, and b3 are blood-related, and c1, c2, and c3 are blood-related. The time of data table generation gradually decreases with the sequence number. The generation time of a1, a2, b1, c1, and c2 is within the time range of t0-t1, and the generation time of a2, a3, b2, b3, c2, and c3 is within the time range of t1-t2; and the a series data tables are adjacent blood relatives of the b series data tables, and the c series data tables are adjacent blood relatives of the b series and are not adjacent blood relatives of the a series. Based on the above relationship, a three-branch data table network can be constructed. The first branch in the network includes data tables a1, a2, and a3, the second branch includes data tables b1, b2, and b3, and the third branch includes c1, c2, and c3. The second branch is adjacent to the first branch and the third branch respectively. If the task to be tested specifies that the test targets the a series data tables and c series data tables, the first branch and the third branch are determined as the target network branches in the data table network. Furthermore, the task to be tested also specifies that the time range of the data table is between t1-t2. Since the generation time of a1, c1, a2, and c2 is within the time range of t0-t1, they are not selected. Therefore, a3 in the first branch and c3 in the third branch are used as the data tables to be tested, completing the acquisition process of the data tables to be tested.

[0097] In the technical solution provided by the embodiment of the present disclosure, there is a clear connection between the data tables in the data table network constructed by the table blood relationship, and each branch plays a role in classifying the data tables; therefore, the data table to be tested corresponding to the task to be tested can be quickly selected from the data table network, thereby improving the efficiency and accuracy of obtaining the data table to be tested.

[0098] In an exemplary embodiment, if Figure 5 As shown, in the above step S320, after obtaining the data table to be tested corresponding to the task to be tested from the data warehouse, the following steps are also included:

[0099] In step S510, metadata information of each data table to be tested is obtained;

[0100] Among them, metadata is data that describes data, mainly information that describes the data attributes of a data table. Based on metadata information, various functions can be realized, such as indicating storage location, historical data, resource search, file records, etc.; metadata can be stored in a data warehouse or a separate metadata database; metadata information can refer to data that defines field attributes in a data table, such as the type and length of a field; usually, different field types have different corresponding character length requirements. For fields of primary key type or foreign key type, they are usually required to be non-empty.

[0101] In step S520, the identifier, attribute and attribute value of the data information in each data table to be tested are obtained according to the metadata information.

[0102] Among them, Figure 6 As shown in the figure, it is a schematic diagram of obtaining the identification, attributes and attribute values ​​of data information through metadata information; the identification is the characteristic information used to identify a certain data information, and can also be in the form of a unique key or feature construction; the attribute refers to a dimension corresponding to the data information, which is the environment for the application of the attribute value, and is used to reflect a type of indicator of business data. The collection of such indicators constitutes a dimension; the attribute value is a clear parameter corresponding to the attribute, for example, if the attribute is the number of users, then the attribute value is 1 million; the identification, attributes and attribute values ​​can be used to identify and classify a certain data information. For example, when performing sampling processing, the distribution of attributes and attribute values ​​can be considered, and the identification can be used as the specific basis for sampling.

[0103] In the technical solution provided by the embodiment of the present disclosure, by acquiring metadata information of the data table to be tested and parsing the identifier, attributes and attribute values ​​of the data information, sampling processing of the data table can be achieved, thereby improving the efficiency of sampling the data table.

[0104] In an exemplary embodiment, the sampling parameters include at least a time range, a sampling quantity, and a sampling method; Figure 7 As shown, in the above step S220, sampling processing is performed on the data information in the data table to obtain at least one sampling table, which can be specifically implemented by the following steps:

[0105] In step S710, data information that meets the time range is filtered out from the data table;

[0106] Specifically, the time range is a limitation on the time for obtaining data information. The time range can obtain data information generated within the corresponding time range from the data table to achieve preliminary screening of the data information.

[0107] In step S720, according to the sampling method, data information of the sampling quantity is extracted from the identifier of the filtered data information as data information of the sampling table.

[0108] Specifically, sampling methods may include random sampling, stratified sampling, etc.; different sampling methods require different sampling parameters; for example, random sampling can randomly select values ​​according to the identifier of the data information, and obtain data information corresponding to the sampling quantity to form a sampling table; stratified sampling can further divide the data information according to different attributes, and sample from each level after the division to obtain data information of the sampling quantity.

[0109] In the technical solution provided by the embodiment of the present disclosure, the data information in the data table can be screened through the time range, and the data information corresponding to the sampling quantity can be obtained according to different sampling methods, so that the data information in the sampling table is more effective, and the obtained sampling table can have the same performance as the data table; that is, the amount of data information in the data table is simplified without losing the data performance of the data table, thereby improving the efficiency of data testing.

[0110] In an exemplary embodiment, the sampling method is stratified sampling; in step S620, according to the sampling method, a sampling quantity of data information is extracted from the identifier of the filtered data information as the data information of the sampling table, and also includes: dividing the filtered data information into multiple levels according to attributes and attribute values; extracting a preset sampling quantity of data information from each level according to the identifier, wherein the preset sampling quantity of data information is the data information of the sampling table.

[0111] Specifically, data information such as data tables can be further classified and divided through attributes and attribute values; the division level can make the sampled data information more evenly distributed. The division level can also unify the data information of multiple sampling tables for comparative analysis. For example, the attributes include users watching live broadcasts and users watching short videos; the attribute information of users watching live broadcasts is the specific duration of watching live broadcasts, and the value distribution ranges from 10 minutes to 110 minutes; the attribute information of users watching short videos is the specific duration of watching live broadcasts, and the value distribution ranges from 20 minutes to 120 minutes. Based on this, if you want to compare the data of product orders placed by two user groups who watch live broadcasts and short videos for the same length of time, you can divide the data information of users watching live broadcasts and users watching short videos into two levels of 20 minutes-60 minutes and 60 minutes-100 minutes according to the specific length of time you watch the live broadcast, and extract the data information of the A1 quantity from the 20 minutes-60 minutes level, and extract the data information of the A2 quantity from the 60 minutes-100 minutes level. A total of A1+A2 quantity, i.e. A3 quantity, is extracted from the two levels and used as the data information of the sampling table. This unifies the levels of the two sampling tables with different attributes so that they can be compared to obtain other features.

[0112] In the technical solution provided by the embodiment of the present disclosure, data information is divided into levels by attribute values, so that the distribution of data information in the sampling table obtained by sampling processing is more even, ensuring that the sampling results can accurately reflect the data characteristics of the data table.

[0113] In an exemplary embodiment, in the above step S220, after performing sampling processing on the data information in the data table to obtain at least one sampling table, it also includes: obtaining a first statistical result of the data table and a second statistical result of at least one sampling table; comparing the first statistical result with the second statistical result to obtain a comparison result; determining the degree of match between the data table and at least one sampling table based on the comparison result; the degree of match is used to determine, from at least one sampling table, a sampling table that is closest to the performance of the data table.

[0114] Among them, the statistical result is a mathematical statistics made based on at least parameter information such as the identifier, attribute and distribution of attribute values ​​of the data information; for example, the enumeration value distribution of each attribute in the data table is obtained for statistics, the magnitude of the identifier of the data information in the data table is removed and then the magnitude is counted, and the attribute values ​​of certain attributes are summed and counted. The first statistical result refers to the sampling result obtained after mathematical statistics are performed on the data table; the second statistical result refers to the statistical result obtained after mathematical statistics are performed on the sampling table. The comparison result refers to the result obtained after comparing the first statistical result with the second statistical result. The comparison result can reflect the similarity and difference between the data table and the sampling table in various dimensions, which is convenient for judging the matching degree between the sampling table and the data table. The number of statistical results contained in the second statistical result corresponds to the number of sampling tables, that is, if only one sampling table is selected for statistics, a second statistical result containing one statistical result is obtained; if three sampling tables are selected for statistics, the second statistical result contains three statistical results; further, when comparing the first statistical result with the second statistical result, if the second statistical result contains statistical results of multiple sampling tables, the comparison result can be multiple results for multiple sampling tables.

[0115] Specifically, after the data table and the corresponding at least one sampling table are statistically analyzed in the same way for the identifiers, attributes and attribute values ​​of the data information contained in each of them to obtain statistical results, the statistical results of the data table and the corresponding at least one sampling table can be compared; since the statistical results are obtained by the same statistical method, the matching degree between the two statistical results can be calculated by a variety of calculation methods, such as extracting the values ​​of certain indicators to calculate the correlation, and the matching degree is determined by the size of the correlation. The matching degree can be used to screen out the sampling table with the closest performance to the data table from the statistical results of at least one sampling table. Since the performance of the sampling table is closest to that of the data table, it can replace the data table in some scenarios, thereby achieving the purpose of simplifying the amount of data in the data table. For example, if the data table Q contains 1 million pieces of data information, it may take a long time for the computer to process it; therefore, the data table Q can be sampled multiple times to obtain multiple sampling tables q1, q2, q3, etc. containing 100,000 pieces of data information; statistics are performed on the data table and multiple sampling tables to obtain statistical results; for example, the data table Q is sampled twice to obtain two data tables q1 and q2, and the selected statistical indicators are "x attribute distribution, time distribution", and the weight of the x attribute distribution is 0.8, and the weight of the time distribution is 0.2; after statistics on the data table Q, the result is: the value of the x attribute is distributed between 0.15-0.36, and the value is 0.20 The number is the largest, and the data information between t3-t4 accounts for 90%; then the same statistics are performed on q1 and q2. If the value of the x attribute in q1 is distributed between 0.15-0.38, and the value of 0.20 is the largest, the data information between t3-t4 accounts for 89%; the value of the x attribute in q2 is distributed between 0.10-0.33, and the value of 0.17 is the largest, and the data information between t3-t4 accounts for 65%; if the second statistical results of q1 and q2 are compared with the first statistical results of Q, it can be clearly found that q1 is the data table with the closest matching degree to Q, which can reflect the data information distribution characteristics of data table Q. The sampling table with the closest matching degree to the data table is used as the sampling table with the closest performance to the data table, that is, the sampling table containing 100,000 data information can already reflect the data characteristics of the data table, so when using it, the data table can be replaced by the sampling table to achieve the purpose of simplifying the amount of data in calculation.

[0116] Specifically, comparative evaluation can also be performed based on the statistical results, and corresponding evaluation reports can be generated and returned to the terminal device; in this way, the user can have a general understanding of the sampling table obtained by sampling. For example, the evaluation report can perform data extraction evaluation from the statistical results based on multiple indicators, generate tables, statistical charts and other data based on the evaluation results, and fill the data according to the preset evaluation report template to obtain the evaluation report.

[0117] Furthermore, since the data table is pre-stored in the data warehouse, when the data table is selected for sampling processing, the server can pre-perform statistics on the data table in a variety of ways, and the statistical process can be performed when the server resources are idle to achieve pre-processing of the data table statistical results. At the same time, it will not occupy system resources when the load is high, thereby improving the efficiency of server data processing.

[0118] In the technical solution provided by the embodiments of the present disclosure, statistics are performed on the identifiers, attributes and attribute values ​​of the data information in the data table and at least one sampling table to obtain statistical results, and the degree of matching between the data table and at least one sampling table is determined based on the statistical results. This makes it possible to select a sampling table with performance closest to the original data table from at least one sampling table, and determine a sampling table with a smaller data volume without losing the performance of the data table, so as to simplify the data volume of the data table.

[0119] In an exemplary embodiment, if Figure 8 As shown, in the above step S230, the test process is executed to obtain a sampling test result corresponding to at least one sampling table, which can be specifically achieved by the following steps:

[0120] In step S810, a preset evaluation index corresponding to the test process is obtained, and the sampling table is evaluated according to the preset evaluation index to obtain an index evaluation result matching the preset evaluation index;

[0121] In step S820, an indicator evaluation report is generated according to the indicator evaluation result; the indicator evaluation report is used to determine the data status of the data table.

[0122] Specifically, the evaluation indicators may include consistency evaluation indicators, statistical evaluation indicators, task execution efficiency evaluation indicators, and custom indicators. Among them, consistency evaluation indicators are evaluations of indicators that are strongly correlated with the upstream data tables of the data table, such as the number of identified items, the sum of a certain attribute value, etc.; statistical evaluation indicators refer to statistical indicators such as standard deviation and extreme value calculated by calculating the statistical results of the outputs of multiple sampling tables after obtaining multiple sampling tables; task execution efficiency evaluation indicators refer to indicators obtained by calculating the task execution time indicator, efficiency, etc. for executing the same task including the data table and multiple sampling tables; custom indicators are evaluation indicators set according to business logic.

[0123] like Fig. 9 , which is a schematic diagram of executing a test process for different numbers of sampling tables in another embodiment;

[0124] Taking the execution of the code to be tested as an example for illustration: First, sample the data table to obtain a sample table; if the number of sample tables is one, replace the data table relied on by the code to be tested with the corresponding sample table; after the replacement is completed, execute the code to be tested, and in the execution process, obtain the corresponding data information from the sample table for calculation and other processes to obtain the test result; write the test result into the sample result table corresponding to the sample table. If the number of sample tables is greater than one, replace the data table relied on by the code to be tested with one of the multiple sample tables respectively; after each replacement is completed, execute the code to be tested until corresponding test results are obtained for all sample tables; write the multiple test results into different sample result tables according to the differences of the corresponding sample tables respectively. The sample result table can be set in the form of "(data table name)_micro_model_(test result of the nth sample table)", for example, "aggr_user_1d_micro_model_1".

[0125] In the technical solution provided by the embodiments of the present disclosure, by generating an index evaluation report according to the index evaluation result, the data situation of the data table can be determined, so as to further adjust and improve the sampling parameters to obtain a sample table in which each evaluation index is close to the original data table.

[0126] In an exemplary embodiment, a method for sampling data information in a data table to obtain at least one sample table is further provided, and the schematic flow diagram is as Fig.10 shown; taking the test process corresponding to the code to be tested as an example, first obtain the data table associated with the code to be tested; determine the time range of the data information, and sample according to the number of sample tables to be obtained. When the number of sample tables is greater than one, sampling with replacement or sampling without replacement can also be selected; sampling without replacement can be implemented by, for example, the statement (rand(hash(key))*m) % n < n * P (rand is a random function, hash is a hash function, key is an identifier, P is the sampling quantity, m is the quantity of data information, n is the number of samplings), and sampling with replacement can be implemented by, for example, the statement (rand()*m) % n < n * P (rand is a random function, P is the sampling quantity, m is the quantity of data information, n is the number of samplings). The obtained sample table can be named in the form of "(sample table name)_sample_(sampling serial number)", for example, A_sample_1, B_sample_1, C_sample_1.

[0127] Determine the sampling quantity and the sampling method, sample the data information in the data table according to different sampling methods in combination with the previously determined sampling parameters, and integrate the sampled data information into a sample table to complete the sampling process.

[0128] In an exemplary embodiment, as Fig.11As shown, another data testing method is provided, comprising:

[0129] Step S1110, in response to a test request including a task to be tested, obtaining the data table lineage relationship of the task to be tested, the data table lineage relationship is used to characterize the upstream and downstream relationship between each data table; according to the data table lineage relationship, obtaining the data table to be tested corresponding to the task to be tested.

[0130] Step S1120, obtaining metadata information of the data table to be tested; obtaining the identifier, attribute and attribute value of the data information in each data table to be tested according to the metadata information.

[0131] Step S1130, filter out data information that meets the time range from the data table; divide the filtered data information into multiple levels according to attributes and attribute values; extract a preset sampling number of data information from each level according to the identifier, wherein the preset sampling number of data information is the data information of the sampling table.

[0132] Step S1140, replace the data table to be tested with at least one sampling table, obtain preset evaluation indicators corresponding to the test process, evaluate the sampling table according to the preset evaluation indicators, and obtain indicator evaluation results that match the preset evaluation indicators; generate an indicator evaluation report based on the indicator evaluation results, and the indicator evaluation report is used to determine the data status of the data table.

[0133] Step S1150, obtaining a first statistical result of the data table and a second statistical result of at least one sampling table; comparing the first statistical result with the second statistical result to obtain a comparison result; determining a matching degree between the data table and at least one sampling table according to the comparison result; the matching degree is used to determine, from at least one sampling table, a sampling table that is closest in performance to the data table.

[0134] The technical solution provided by the embodiment of the present disclosure achieves the goal of obtaining at least one sampling table with less data information by sampling the data table, thereby reducing the amount of data in the data testing process and improving data processing efficiency; at the same time, at least one sampling table is used to execute the test process, and the test results of the data table are determined based on multiple sampling test results, which also improves the comprehensiveness and accuracy of data testing on the data table and further improves the efficiency of data testing.

[0135] It should be understood that although Figure 2 , 3 The steps in the flowcharts of 5-11 are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 ,3 , at least part of the steps in 5-11 may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0136] It can be understood that the same / similar parts between the various embodiments of the above method in this specification can refer to each other, and each embodiment focuses on the differences from other embodiments. For related points, please refer to the description of other method embodiments.

[0137] Fig.12 FIG. 1 is a block diagram of a data testing device according to an exemplary embodiment. Fig.12 The device includes a data table acquisition unit 1202, a sampling table acquisition unit 1204, a data testing unit 1206 and a result acquisition unit 1208.

[0138] The data table acquisition unit 1202 is configured to acquire a data table to be tested, where the data table is a collection of data information required in a test process.

[0139] The sampling table acquisition unit 1204 is configured to perform sampling processing on the data information in the data table to obtain at least one sampling table, and the sampling table is used to store the sampled data information in the data table.

[0140] The data testing unit 1206 is configured to replace the data table to be tested with at least one sampling table, and execute the test process to obtain a sampling test result corresponding to the at least one sampling table.

[0141] The result acquisition unit 1208 is configured to determine a test result of the data table according to at least one sample test result.

[0142] In an exemplary embodiment, the data table acquisition unit 1202 is also configured to execute in response to a test request containing a task to be tested, obtain the data table lineage relationship of the task to be tested, and the data table lineage relationship is used to characterize the upstream and downstream relationship between each data table; according to the data table lineage relationship, obtain the data table to be tested corresponding to the task to be tested.

[0143] In an exemplary embodiment, the data table acquisition unit 1202 is further configured to construct a data table network according to the data table blood relationship, determine the target network branch in the data table network; and select at least one data table from each target network branch as the data table to be tested corresponding to the task to be tested.

[0144] In an exemplary embodiment, the data table acquisition unit 1202 is further configured to acquire metadata information of each data table to be tested; and acquire the identifier, attribute and attribute value of the data information in each data table to be tested according to the metadata information.

[0145] In an exemplary embodiment, the sampling parameters include at least a time range, a sampling quantity, and a sampling method; the sampling table acquisition unit 1204 is further configured to filter out data information that meets the time range from the data table; and according to the sampling method, extract the data information of the sampling quantity from the identifier of the filtered data information as the data information of the sampling table.

[0146] In an exemplary embodiment, the sampling method is stratified sampling; the sampling table acquisition unit 1204 is also configured to divide the filtered data information into multiple levels according to attributes and attribute values; and extract a preset sampling quantity of data information from each level according to the identifier, wherein the preset sampling quantity of data information is the data information of the sampling table.

[0147] In an exemplary embodiment, the sampling table acquisition unit 1204 is further configured to execute the acquisition of a first statistical result of the data table and a second statistical result of at least one sampling table; compare the first statistical result with the second statistical result to obtain a comparison result; determine the degree of match between the data table and at least one sampling table based on the comparison result; the degree of match is used to determine, from at least one sampling table, a sampling table that is closest to the performance of the data table.

[0148] In an exemplary embodiment, the data testing unit 1206 is also configured to execute and obtain preset evaluation indicators corresponding to the test process, evaluate the sampling table according to the preset evaluation indicators, obtain indicator evaluation results that match the preset evaluation indicators, and generate an indicator evaluation report according to the indicator evaluation results; the indicator evaluation report is used to determine the data status of the data table.

[0149] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0150] Fig.13 1 is a block diagram of an electronic device 1300 for data testing according to an exemplary embodiment. For example, the electronic device 1300 may be a server. Fig.13, the electronic device 1300 includes a processing component 1320, which further includes one or more processors, and a memory resource represented by a memory 1322 for storing instructions that can be executed by the processing component 1320, such as an application. The application stored in the memory 1322 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1320 is configured to execute the instructions to perform the above method.

[0151] The electronic device 1300 may further include a power supply component 1324 configured to perform power management of the electronic device 1300, a wired or wireless network interface 1326 configured to connect the electronic device 1300 to a network, and an input / output (I / O) interface 1328. The electronic device 1300 may operate based on an operating system stored in the memory 1322, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, or the like.

[0152] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 1322 including instructions, and the above instructions can be executed by a processor of the electronic device 1300 to perform the above method. The storage medium can be a computer-readable storage medium, for example, a computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0153] In an exemplary embodiment, a computer program product is further provided. The computer program product includes instructions, and the instructions can be executed by a processor of the electronic device 1300 to complete the above method.

[0154] It should be noted that the above-mentioned devices, electronic devices, computer-readable storage media, computer program products, etc. may also include other implementation methods according to the description of the method embodiments. The specific implementation methods can refer to the description of the relevant method embodiments, which will not be described one by one here.

[0155] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0156] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A data testing method, characterized in that: include: Obtain a data table to be tested, wherein the data table is a collection of data information required in the test process; Performing sampling processing on the data information in the data table to obtain at least one sampling table, wherein the sampling table is used to store the data information sampled from the data table; different sampling tables contain different amounts of data information according to different sampling parameters; the sampling parameters at least include a time range, a sampling quantity, and a sampling method; The data table to be tested is replaced by the at least one sampling table, and the test process is executed to obtain a sampling test result corresponding to the at least one sampling table; the sampling test results of each sampling table are used to reflect the test results of the data table; In the case where at least one of the sampling test results is multiple, at least one of the sampling test results is used to reflect the data characteristics of the data table; Determine the test result of the data table according to at least one of the sampling test results; The test result is a test result obtained by integrating at least one of the sample test results; The step of replacing the data table to be tested with the at least one sampling table comprises: The data table to be tested is replaced with multiple sampling tables; or, the data table to be tested is replaced with a sampling table whose performance is closest to the data table; the sampling table whose performance is closest to the data table is determined from the at least one sampling table based on the degree of matching between the data table and the at least one sampling table.

2. The method according to claim 1, characterized in that The step of obtaining the data table to be tested includes: In response to a test request including a task to be tested, obtaining a data table lineage relationship of the task to be tested, wherein the data table lineage relationship is used to characterize an upstream and downstream relationship between various data tables; According to the data table lineage relationship, the data table to be tested corresponding to the task to be tested is obtained.

3. The method according to claim 2, characterized in that The step of obtaining the data table to be tested corresponding to the task to be tested according to the data table lineage relationship includes: Constructing a data table network according to the data table blood relationship, and determining a target network branch in the data table network; At least one data table is selected from each of the target network branches as the data table to be tested corresponding to the task to be tested.

4. The method according to claim 2, characterized in that: After obtaining the data table to be tested corresponding to the task to be tested, the method further includes: Obtaining metadata information of each of the data tables to be tested; According to the metadata information, the identifier, attribute and attribute value of the data information in each of the data tables to be tested are obtained.

5. The method according to claim 4, characterized in that The performing sampling processing on the data information in the data table to obtain at least one sampling table includes: Filtering data information that meets the time range from the data table; According to the sampling method, data information of the sampling quantity is extracted from the identifier of the screened data information as data information of the sampling table.

6. The method according to claim 5, characterized in that The sampling method described is stratified sampling; The method of extracting the data information of the sampling quantity from the identifier of the filtered data information according to the sampling method as the data information of the sampling table includes: According to the attributes and the attribute values, the filtered data information is divided into multiple levels; According to the identifier, a preset sampling quantity of data information is extracted from each level respectively, wherein the preset sampling quantity of data information is the data information of the sampling table.

7. The method according to claim 4, characterized in that After performing sampling processing on the data information in the data table to obtain at least one sampling table, the method further includes: Obtaining a first statistical result of the data table and a second statistical result of the at least one sampling table; Comparing the first statistical result with the second statistical result to obtain a comparison result; The matching degree between the data table and the at least one sampling table is determined according to the comparison result; the matching degree is used to determine the sampling table with the closest performance to the data table from the at least one sampling table.

8. The method according to claim 1, characterized in that The executing the test process to obtain a sampling test result corresponding to the at least one sampling table includes: Obtaining a preset evaluation indicator corresponding to the test process, evaluating the sampling table according to the preset evaluation indicator, and obtaining an indicator evaluation result matching the preset evaluation indicator; An indicator evaluation report is generated according to the indicator evaluation result, and the indicator evaluation report is used to determine the data status of the data table.

9. A data testing device, characterized in that: include: A data table acquisition unit is configured to acquire a data table to be tested, wherein the data table is a collection of data information required in the test process; A sampling table acquisition unit is configured to perform sampling processing on the data information in the data table to obtain at least one sampling table, wherein the sampling table is used to store the data information sampled from the data table; different sampling tables contain different amounts of data information according to different sampling parameters; the sampling parameters at least include a time range, a sampling quantity, and a sampling method; A data testing unit, configured to replace the data table to be tested with the at least one sampling table, and execute the test process to obtain a sampling test result corresponding to the at least one sampling table; the sampling test results of each sampling table are used to reflect the test results of the data table; In the case where at least one of the sampling test results is multiple, at least one of the sampling test results is used to reflect the data characteristics of the data table; A result acquisition unit, configured to determine a test result of the data table according to at least one of the sample test results; The test result is a test result obtained by integrating at least one of the sample test results; The data testing unit is further configured to replace the data table to be tested with a plurality of sampling tables; or replace the data table to be tested with a sampling table that is closest in performance to the data table; The sampling table that is closest in performance to the data table is determined from the at least one sampling table according to the degree of matching between the data table and the at least one sampling table.

10. The device according to claim 9, characterized in that The data table acquisition unit is also configured to execute in response to a test request containing a task to be tested, obtain the data table lineage relationship of the task to be tested, and the data table lineage relationship is used to characterize the upstream and downstream relationship between each data table; according to the data table lineage relationship, obtain the data table to be tested corresponding to the task to be tested.

11. The device according to claim 10, characterized in that The data table acquisition unit is further configured to construct a data table network according to the data table blood relationship, determine the target network branch in the data table network; and select at least one data table from each of the target network branches as the data table to be tested corresponding to the task to be tested.

12. The device according to claim 10, characterized in that The data table acquisition unit is further configured to acquire metadata information of each of the data tables to be tested; and acquire identifiers, attributes and attribute values ​​of data information in each of the data tables to be tested according to the metadata information.

13. The device according to claim 12, characterized in that The sampling table acquisition unit is further configured to filter out data information that meets the time range from the data table; and extract the data information of the sampling quantity from the identifier of the filtered data information according to the sampling method as the data information of the sampling table.

14. The device according to claim 13, characterized in that The sampling method described is stratified sampling; The sampling table acquisition unit is further configured to divide the filtered data information into multiple levels according to the attributes and the attribute values; and extract a preset sampling quantity of data information from each level according to the identifier, wherein the preset sampling quantity of data information is the data information of the sampling table.

15. The device according to claim 12, characterized in that The sampling table acquisition unit is further configured to execute acquisition of a first statistical result of the data table and a second statistical result of the at least one sampling table; compare the first statistical result with the second statistical result to obtain a comparison result; Determining a matching degree between the data table and the at least one sampling table according to the comparison result; The matching degree is used to determine a sampling table with performance closest to the data table from at least one sampling table.

16. The device according to claim 9, characterized in that The data testing unit is also configured to execute and obtain preset evaluation indicators corresponding to the test process, evaluate the sampling table according to the preset evaluation indicators, obtain indicator evaluation results that match the preset evaluation indicators, and generate an indicator evaluation report according to the indicator evaluation results; the indicator evaluation report is used to determine the data status of the data table.

17. A server, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the data testing method according to any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of a server, the server is enabled to execute the data testing method according to any one of claims 1 to 8.

19. A computer program product, comprising instructions, characterized in that: When the instruction is executed by a processor of the server, the server is enabled to execute the data testing method according to any one of claims 1 to 8.

Citation Information

Patent Citations

  • Data automation test method and apparatus

    CN105335407A

  • Data cleaning method, device and equipment

    CN109977110A

  • Method and system for extracting sample data from big data

    CN110362562A