A data warehouse testing method, device and equipment

Abstract

The present application discloses a method, device and equipment for testing a data warehouse, which can be applied to distributed fields or financial fields. The method includes: obtaining a data warehouse model to be tested corresponding to the data warehouse to be tested; obtaining metadata information of the data warehouse to be tested according to the data warehouse model to be tested; obtaining blood relationship information of the data warehouse to be tested according to the data warehouse model to be tested; inputting test data into the data warehouse to be tested, so that the data warehouse to be tested is scheduled and generates data for each layer of model tables; splicing a test script according to the metadata information and / or the blood relationship information, so that each test point of the data warehouse to be tested has a corresponding test script for testing; executing the test script for the data of each layer of model tables to generate test results. In this way, each test point of the data warehouse has a corresponding test script for automatic testing, which improves the test efficiency of comprehensive testing of the data warehouse.

Description

Technical Field

[0001] The present application relates to the technical field of automated testing, and in particular to a testing method, device and equipment for a data warehouse. Background Art

[0002] With the rapid development of enterprises, more and more systems are used, so a large number of different systems are running at the same time. It is difficult for these systems to provide unified reports, and the support and licensing costs are rising. Over time, data warehouses have emerged, but data warehouses need to be maintained so that they can work stably. At present, how to better and more comprehensively maintain data warehouses through testing has become the key.

[0003] A data warehouse is a subject-oriented, integrated, time-varying, but relatively stable data set used to support management decision-making processes. The data warehouse itself does not "produce" any data, nor does it need to "consume" any data. The data comes from the outside and is open to external applications. The existing technology for testing data warehouses often requires testers to manually analyze the mapping relationship of fields one by one and write corresponding test scripts for testing, which makes the test efficiency low and difficult to fully cover.

[0004] Therefore, how to improve the testing efficiency of comprehensive testing of data warehouses is an issue that technical personnel in this field urgently need to solve. Summary of the invention

[0005] Based on the above problems, the present application provides a testing method for a data warehouse. By splicing metadata information and / or blood relationship information into test scripts and configuring relevant test information, each test point in the data warehouse has a corresponding test script for testing, thereby solving the problem of low testing efficiency and insufficient comprehensiveness when testing data warehouses in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a data warehouse testing method, including:

[0007] Obtain the data warehouse model to be tested corresponding to the data warehouse to be tested;

[0008] According to the data warehouse model to be tested, metadata information of the data warehouse to be tested is obtained;

[0009] According to the data warehouse model to be tested, obtaining the blood relationship information of the data warehouse to be tested;

[0010] Input the test data into the data warehouse to be tested, so that the data warehouse to be tested performs scheduling and generates data of each layer model table;

[0011] Splicing a test script according to the metadata information and / or the blood relationship information so that each test point of the data warehouse to be tested has a corresponding test script for testing;

[0012] The test script is executed for the data in the model tables of each layer to generate test results.

[0013] Optionally, acquiring metadata information of the data warehouse to be tested according to the data warehouse model to be tested includes:

[0014] According to the data warehouse model to be tested, obtaining each model table of each layer of the data warehouse to be tested;

[0015] Obtain metadata information of each model table of each layer of the data warehouse to be tested.

[0016] Optionally, obtaining the blood relationship information of the data warehouse to be tested according to the data warehouse model to be tested includes:

[0017] According to the data warehouse model to be tested, obtaining each model table of each layer of the data warehouse to be tested;

[0018] Obtain the blood relationship between each model table of each layer of the data warehouse to be tested.

[0019] Optionally, the method further includes:

[0020] constructing test data according to the metadata information;

[0021] The step of inputting the test data into the data warehouse to be tested, so that the data warehouse to be tested performs scheduling and generates data of each layer model table, includes:

[0022] The test data is input into the data warehouse to be tested, and the data warehouse to be tested is scheduled according to the configuration information to generate data of each layer model table.

[0023] Optionally, the method further includes:

[0024] Acquire production bleaching data, and use the production bleaching data as test data;

[0025] The step of inputting the test data into the data warehouse to be tested, scheduling and generating data of the physical tables of each layer model according to the data warehouse to be tested includes:

[0026] The test data is input into the data warehouse to be tested, and the data warehouse to be tested is scheduled according to the configuration information to generate data of each layer model table.

[0027] Optionally, the step of splicing a test script according to the metadata information and / or the blood relationship information so that each test point of the data warehouse to be tested has a corresponding test script for testing includes:

[0028] The configuration information is used to assemble a test script according to the metadata information and / or the blood relationship information, so that each test point of the data warehouse to be tested has a corresponding test script for testing.

[0029] Optionally, the configuration information is used to splice a test script according to the metadata information and / or the blood relationship information so that each test point of the data warehouse to be tested has a corresponding test script for testing, including:

[0030] The configuration information is used to assemble a test script according to the metadata information, so that the metadata integrity test point, primary key uniqueness test point, primary key not empty test point, zipper table logic test point, and date field test point of the data warehouse to be tested have corresponding test scripts for testing;

[0031] Using the configuration information to assemble a test script according to the blood relationship information, so that the data warehouse record number integrity test point to be tested has a corresponding test script for testing;

[0032] The configuration information is used to assemble a test script according to the metadata information and the blood relationship information, so that the primary and foreign key relationship integrity test points of the data warehouse to be tested and the source table to target table mapping logic test points have corresponding test scripts for testing.

[0033] Optionally, the method further includes:

[0034] Using the configuration information, a test script is assembled according to the standard code table designed by the model in the project and the mapping relationship between the code conversion fields in the data warehouse, so that the standard code conversion test point of the data warehouse to be tested is tested using the test script;

[0035] The configuration information is used to assemble a test script according to the clear total-to-score relationship contained in the business involved in the data warehouse to be tested, so that the total-to-score relationship continuity test point of the data warehouse to be tested is tested using the test script.

[0036] In a second aspect, an embodiment of the present application provides a data warehouse testing device, including:

[0037] An acquisition module is used to acquire the data warehouse model to be tested corresponding to the data warehouse to be tested;

[0038] A metadata information module, used to obtain metadata information of the data warehouse to be tested according to the data warehouse model to be tested;

[0039] A blood relationship information module, used to obtain blood relationship information of the data warehouse to be tested according to the data warehouse model to be tested;

[0040] A data preparation module, used for inputting test data into the data warehouse to be tested, so that the data warehouse to be tested performs scheduling and generates data for each layer model table;

[0041] The automated testing module is used to assemble a test script based on the metadata information and / or the blood relationship information, so that each test point in the data warehouse to be tested has a corresponding test script for testing; the test script is executed for the data in the model tables of each layer to generate test results.

[0042] In a third aspect, an embodiment of the present application provides a data warehouse testing device, including:

[0043] Memory for storing computer programs;

[0044] A processor, configured to implement the steps of the method for testing a data warehouse as claimed in any one of claims 1 to 8 when executing the computer program.

[0045] It can be seen from the above technical solutions that compared with the prior art, the present application has the following advantages:

[0046] After obtaining the data warehouse model to be tested corresponding to the data warehouse to be tested, the present application obtains the metadata information of the data warehouse to be tested and the blood relationship information of the data warehouse to be tested according to the data warehouse model to be tested. Then the test data is input into the data warehouse to be tested, so that the data warehouse to be tested is scheduled and generates data for each layer of the model table. Afterwards, a test script is spliced ​​into the metadata information and / or the blood relationship information, so that each test point of the data warehouse to be tested has a corresponding test script for testing, and finally the test script is executed for the data of each layer of the model table to generate the test results. In this way, each test point of the data warehouse has a corresponding test script for automatic testing, which improves the test efficiency of the comprehensive test of the data warehouse. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A flowchart of a data warehouse testing method provided in an embodiment of the present application;

[0048] Figure 2 A specific automated testing module is provided for the embodiment of the present application;

[0049] Figure 3 A specific application scenario is provided for the embodiments of the present application;

[0050] Figure 4 A schematic diagram of the structure of a data warehouse testing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] As mentioned above, the existing technology for testing data warehouses is often inefficient and difficult to fully cover. Specifically, due to the different model design strategies adopted by various enterprises when establishing data warehouse models and the different business areas of various enterprises, the current data warehouse testing lacks corresponding industry standards and testing methods, and there are no universal testing tools. When testing data warehouse projects, the internal data logic and mapping of the data model involved are very complex. If you want to test it completely, the tester needs to manually analyze the mapping relationship of the fields one by one and write the corresponding test scripts for testing, which makes it difficult to ensure test efficiency and full coverage.

[0052] In order to solve the above problems, the present application provides a method for testing a data warehouse, which includes: after obtaining a data warehouse model to be tested corresponding to the data warehouse to be tested, obtaining metadata information of the data warehouse to be tested according to the data warehouse model to be tested, and obtaining blood relationship information of the data warehouse to be tested according to the data warehouse model to be tested. Then, the test data is input into the data warehouse to be tested, so that the data warehouse to be tested is scheduled and generates data for each layer of model tables. Afterwards, a test script is spliced ​​into the metadata information and / or blood relationship information, so that each test point of the data warehouse to be tested has a corresponding test script for testing, and finally, the test script is executed for the data of each layer of model tables to generate test results.

[0053] In this way, the metadata information and / or blood relationship information are spliced ​​into test scripts, so that each test point in the data warehouse has a corresponding test script for automatic testing, thereby improving the test efficiency of comprehensive testing of the data warehouse.

[0054] It should be noted that the test method, device and equipment for a data warehouse provided in this application can be used in the fields of artificial intelligence, blockchain, distributed computing, cloud computing, big data, the Internet of Things, mobile Internet, network security, chips, virtual reality, augmented reality, holographic technology, quantum computing, quantum communication, quantum measurement, digital twins or finance. The above is only an example and does not limit the application field of the test method, device and equipment for a data warehouse provided in this application.

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] Figure 1 A flowchart of a data warehouse testing method provided in an embodiment of the present application. Figure 1 As shown, a data warehouse testing method provided in an embodiment of the present application may include:

[0057] S101: Obtain a data warehouse model to be tested corresponding to the data warehouse to be tested.

[0058] In practical applications, a data warehouse is a subject-oriented, integrated, time-varying, but relatively stable data set used to support management decision-making processes. The data warehouse itself does not "produce" any data, nor does it need to "consume" any data. The data comes from the outside and is open to external applications. In the development of data warehouses, most of the time, technical personnel first establish a data warehouse model, and then implement the establishment of the data warehouse based on the data warehouse model. In other words, there is a corresponding relationship between the data warehouse and the data warehouse model. In the embodiment provided in the present application, if you want to implement a test of the data warehouse to be tested, you need to first obtain the data warehouse model to be tested corresponding to the data warehouse to be tested.

[0059] S102: According to the data warehouse model to be tested, metadata information of the data warehouse to be tested is obtained.

[0060] In practical applications, metadata is also called intermediary data or relay data. It is data that describes data, mainly information that describes data attributes, and is used to support functions such as indicating storage location, historical data, resource search, and file records. In the field of data warehouses, metadata is divided into technical metadata and business metadata according to their purpose. First, metadata can provide user-based information, such as metadata that records business description information of data items to help users use data. Secondly, metadata can support the system's management and maintenance of data, such as metadata about data item storage methods can support the system to access data in the most efficient way. Because there is a corresponding relationship between the data warehouse to be tested and the data warehouse model to be tested, in the embodiments provided in the present application, information describing data attributes in the data warehouse to be tested, that is, metadata information of the data warehouse to be tested, can be obtained based on the data warehouse model to be tested. For example, gender, color, shape, etc.

[0061] In addition, since the storage location of the metadata information is different, the corresponding acquisition method is also different, so the present application can explain the possible acquisition method.

[0062] In one case, metadata information is obtained from each model table of each layer of the data warehouse to be tested. Accordingly, S102 may specifically include:

[0063] According to the data warehouse model to be tested, obtaining each model table of each layer of the data warehouse to be tested;

[0064] Obtain metadata information of each model table of each layer of the data warehouse to be tested.

[0065] In practical applications, in order to have a clear data structure and facilitate data lineage tracking, a data warehouse needs to have a multi-layer structure. For data warehouses, each enterprise will be divided into different layers according to its own business needs, but the most basic layering idea is theoretically divided into three layers, data operation layer, data warehouse layer and data service layer, and each layer is composed of multiple model tables. New layers are added based on this basic layering to meet different business needs. In the embodiment provided in the present application, based on the correspondence between the data warehouse model to be tested and the data warehouse to be tested, according to the data warehouse model to be tested, the model tables of each layer of the data warehouse to be tested are obtained, and then the metadata information is obtained from the model tables of each layer of the acquired data warehouse to be tested.

[0066] S103: Acquire the blood relationship information of the data warehouse to be tested according to the data warehouse model to be tested.

[0067] In practical applications, data lineage refers to the origin and development of data, which mainly includes the source of data, data processing methods, mapping relationships and data exports. Usually we will perform various processing on the original data in multiple steps, and finally generate new data. In this process, many tables will be generated. The link relationship between these data tables can be called data lineage. The lineage relationship in the data warehouse refers to the link relationship and mapping relationship between the model tables at each layer of the data warehouse from the source table. Because there is a corresponding relationship between the data warehouse to be tested and the data warehouse model to be tested, in the embodiments provided in the present application, the lineage information in the data warehouse to be tested can be obtained according to the data warehouse model to be tested.

[0068] In addition, since the objects corresponding to the blood relationship are different, the blood relationship obtained may also be different. This application describes the blood relationship that may be obtained.

[0069] In one case, in order to obtain the blood relationship between each model table of each layer of the data warehouse to be tested, S103 may specifically include:

[0070] According to the data warehouse model to be tested, obtaining each model table of each layer of the data warehouse to be tested;

[0071] Obtain the blood relationship between each model table of each layer of the data warehouse to be tested.

[0072] In practical applications, because the data warehouse is theoretically divided into three layers, the data operation layer, the data warehouse layer and the data service layer, and each layer is composed of multiple model tables. New layers are added based on this basic layering to meet different business needs. Therefore, in the embodiment provided by the present application, based on the correspondence between the data warehouse model to be tested and the data warehouse to be tested, the model tables of each layer of the data warehouse to be tested are obtained according to the data warehouse model to be tested, and then the blood relationship between each model table of each layer of the data warehouse to be tested is obtained according to the obtained model tables of each layer of the data warehouse to be tested.

[0073] S104: Input the test data into the data warehouse to be tested, so that the data warehouse to be tested is scheduled and generates data of each layer model table.

[0074] In practical applications, if you want to test the data warehouse, you need to ensure that the data warehouse contains specific data for testing. In the embodiment provided in this application, the test data needs to be input into the data warehouse to be tested, and the scheduling of the test data is realized through the scheduling function of the data warehouse, so as to generate data of each layer model table.

[0075] In addition, due to different test data acquisition methods, the corresponding test data acquired is also different. Moreover, how the data warehouse generates the data of each layer of model tables is also different, so this application can explain the possible methods of acquiring test data and generating data of each layer of model tables.

[0076] In one case, the test data is obtained according to the metadata information, and the data of each layer model table is generated according to the configuration information. Accordingly, the method further includes:

[0077] constructing test data according to the metadata information;

[0078] The step of inputting the test data into the data warehouse to be tested, so that the data warehouse to be tested performs scheduling and generates data of each layer model table, includes:

[0079] The test data is input into the data warehouse to be tested, and the data warehouse to be tested is scheduled according to the configuration information to generate data of each layer model table.

[0080] In practical applications, metadata information is data that describes data attributes, such as gender, color, shape, etc. We can construct some test data through these data that describe data attributes, for example, specifically, gender is female, color is blue, shape is square, etc. And the configuration information is used to assist in generating data for each layer of model tables, which may include the data date to be scheduled for this test, the database address of the data warehouse to be tested, etc. For example, the configuration data date is January 1, 2022, and then the specific test data "female dressed in blue" constructed by the "gender" and "color" of the metadata information is input into the data warehouse to be tested, and then the test data is scheduled according to the configuration information. When the data of each layer of model tables is finally generated, the data of each layer of model tables on January 1, 2022 will be generated, that is, the female dressed in blue on January 1, 2022. Therefore, in the embodiment provided in the present application, specific test data is first constructed according to the obtained metadata information, and then the test data is input into the data warehouse to be tested. At this time, the data warehouse will schedule the test data according to the configuration information to generate more specific data for each layer of model tables.

[0081] In one case, the test data is obtained from the production bleaching data, and the data of each layer model table is generated according to the configuration information. Accordingly, the method further includes:

[0082] Acquire production bleaching data, and use the production bleaching data as test data;

[0083] The step of inputting the test data into the data warehouse to be tested, scheduling and generating data of the physical tables of each layer model according to the data warehouse to be tested includes:

[0084] The test data is input into the data warehouse to be tested, and the data warehouse to be tested is scheduled according to the configuration information to generate data of each layer model table.

[0085] In practical applications, production bleached data is data used in production, from which sensitive information is removed by data transformation. In the embodiments provided in the present application, production bleached data can be obtained, and then the production bleached data is processed as test data. Specifically, the production bleached data is input as test data into the data warehouse to be tested. At this time, the data warehouse will schedule the test data according to the configuration information to generate more specific data of each layer model table.

[0086] S105: Splicing a test script according to the metadata information and / or the blood relationship information so that each test point of the data warehouse to be tested has a corresponding test script for testing.

[0087] In actual applications, there are many test points that need to be tested for the testing of data warehouses. For the testing of these test points, corresponding test scripts need to be used for testing, and the methods of obtaining these test scripts are different. In the embodiment provided by the present application, by splicing the obtained metadata information and / or the blood relationship information, the test scripts corresponding to each test point of the data warehouse to be tested can be obtained for testing.

[0088] In addition, since the methods of splicing the test scripts according to the metadata information and / or the blood relationship information are different, accordingly, S105 specifically includes:

[0089] The configuration information is used to assemble a test script according to the metadata information and / or the blood relationship information, so that each test point of the data warehouse to be tested has a corresponding test script for testing.

[0090] In actual applications, configuration information is used to assist in generating test scripts, and may include time, method, and so on. For example, if a test point in a data warehouse needs to be tested on January 1, 2022, the time configuration information January 1, 2022 can be spliced ​​into the test script so that the test script includes the running time of the script. In the embodiment provided in the present application, configuration information including information such as time is used to splice into a test script according to the metadata information and / or the blood relationship information, so that each test point in the data warehouse to be tested has a corresponding test script for testing.

[0091] In addition, since the testing of the test warehouse includes the testing of many test points, and the test scripts required for these test points are not the same, the present application can provide a description of possible test point tests.

[0092] In one case, for testing different test points, the configuration information is used to splice the test script according to the metadata information and / or the blood relationship information, so that each test point of the data warehouse to be tested has a corresponding test script for testing, specifically including:

[0093] The configuration information is used to assemble a test script according to the metadata information, so that the metadata integrity test point, primary key uniqueness test point, primary key not empty test point, zipper table logic test point, and date field test point of the data warehouse to be tested have corresponding test scripts for testing;

[0094] Using the configuration information to assemble a test script according to the blood relationship information, so that the data warehouse record number integrity test point to be tested has a corresponding test script for testing;

[0095] The configuration information is used to assemble a test script according to the metadata information and the blood relationship information, so that the primary and foreign key relationship integrity test points of the data warehouse to be tested and the source table to target table mapping logic test points have corresponding test scripts for testing.

[0096] In practical applications, the automated testing module can classify different types of tests through its submodules. Figure 2As shown, specifically, the automated test module can have a general test module to test general test points and a customized test module to test data warehouse items that need customized testing. The tests of general test points include: metadata integrity test, record number integrity test, primary key uniqueness test, primary key not null test, primary and foreign key relationship integrity test, zipper table logic test, date field test and source table to target table mapping logic test. The customized test module includes: standard code conversion test and total score relationship continuity test.In the embodiments provided in the present application, tests adapted to all data warehouse projects, i.e., general test point tests, include metadata integrity tests (testing the table structure of the physical table built in the data warehouse, including whether the field name, field type, field length, etc. are complete and correct), record number integrity tests (testing whether the actual number of records in each model table in the data warehouse meets expectations. For example, through blood relationship, it is concluded that the model table in the data warehouse to be tested is a summary of records from several tables in the lower layer, then the record number relationship is that the sum of the record numbers of several tables in the lower layer is equal to the record number of the table to be tested, and then a test script is generated by splicing for testing; or according to blood relationship, The relationship can be concluded that the records of the table to be tested are the summary of the number of records of some tables in its lower-level model after filtering by certain conditions. The relationship of the number of records is that the sum of the number of records of some tables in the lower layer minus the number of records that meet the filtering conditions is equal to the number of records of the table to be tested, and then the test script is generated by splicing for testing), primary key uniqueness test (test whether the primary key data of each table in the data warehouse is unique, for example, by splicing the metadata information (primary key information) of each model table in each layer of the data warehouse to generate a test script for testing), primary key is not empty test (test that the primary key data of each table in the data warehouse does not have empty values), primary and foreign key relationship integrity Test (test whether the foreign key values ​​of each table in the data warehouse exist in the corresponding main table, that is, the foreign key values ​​in the foreign key table are a subset of the primary key values ​​in the main table, for example, by splicing the primary key and foreign key information of each model table in each layer of the data warehouse and the primary and foreign key relationships between the main table and the sub-table to generate a test script for testing), zipper table logic test (all zipper tables in the data warehouse do not have broken links (END_DT (end time) of the previous state and START_DT (start time) of the next state cannot be connected), reverse chains (START_DT and END_DT are written in reverse) and cross chains (END of the previous state is written in reverse) _DT is greater than or equal to the next state's END_DT), date field test (test whether the date field format in the data warehouse is converted according to the correct date format), source table to target table mapping logic test (test whether each source table field is loaded into the data warehouse according to the designed mapping rules. For example, for two-way comparison, it is necessary to re-implement the data warehouse extraction, conversion, and loading process according to the model mapping rules, and then perform field data comparison through the test script). The test scripts used for each test item are different, but the splicing method of the test scripts corresponding to some test items is the same. In the embodiments provided in the present application, for the testing of the metadata integrity test points, primary key uniqueness test points, primary key not empty test points, zipper table logic test points, and date field test points of the data warehouse to be tested, it is necessary to use the configuration information to splice them into corresponding test scripts according to the metadata information to provide a test basis.For the test of the integrity test point of the number of records in the data warehouse to be tested, it is necessary to use the configuration information to splice the corresponding test scripts according to the lineage information to provide a test basis. For the test of the integrity test point of the primary and foreign key relationship of the data warehouse to be tested and the test point of the mapping logic from the source table to the target table, it is necessary to use the configuration information to splice the metadata information and lineage information into the corresponding test scripts to provide a test basis.

[0097] In addition, due to the differences in the projects in the data warehouse, the corresponding test points that need to be tested in the projects in different data warehouses are also different. That is, it includes test points that are applicable to all data warehouse projects and test points that need to be configured using the characteristics of specific data warehouse projects. Therefore, this application can explain the possible test points that need to be configured using the characteristics of specific data warehouse projects.

[0098] In one case, the test point test needs to be configured using the characteristics of a specific data warehouse project. Accordingly, the method further includes:

[0099] Using the configuration information, a test script is assembled according to the standard code table designed by the model in the project and the mapping relationship between the code conversion fields in the data warehouse, so that the standard code conversion test point of the data warehouse to be tested is tested using the test script;

[0100] The configuration information is used to assemble a test script according to the clear total-to-score relationship contained in the business involved in the data warehouse to be tested, so that the total-to-score relationship continuity test point of the data warehouse to be tested is tested using the test script.

[0101] In actual applications, data warehouse projects that require customized testing also need to be tested. Generally, configuration testing is required based on the characteristics of specific data warehouse projects, including standard code conversion testing (testing whether there is data other than standard code values ​​in the field values ​​involved in code value conversion. For example, the mapping relationship between the standard code table designed in the model in the project and the code conversion field in the data warehouse is spliced ​​to generate a test script for testing, where the code may include: industry type code, currency code, etc.), total score relationship continuity testing (testing whether the total score relationship of the actual data of the field is established. For example, by splicing the model table information and field information containing the total score relationship to generate a test script for testing, where the total score relationship can be a total score relationship in time, a total score relationship in organization, a total score relationship in product type, a total score relationship in currency, etc.). Similarly, the test scripts used for each test project are different, and the splicing method of the test scripts corresponding to such test projects is also different. In the embodiment provided in the present application, for the test of the standard code conversion test point of the data warehouse to be tested, it is necessary to use the configuration information to splice the standard code table designed in the model in the project and the mapping relationship of the code conversion fields involved in the data warehouse into a corresponding test script to provide a test basis. For the test of the continuity test point of the total-score relationship of the data warehouse to be tested, it is necessary to use the configuration information to splice the corresponding test script according to the clear total-score relationship contained in the business involved to provide a test basis.

[0102] S106: Execute the test script for the data in the model tables of each layer to generate test results.

[0103] In practical applications, the test of the data warehouse to be tested requires the data of each layer of model tables and the test scripts corresponding to each test point to cooperate with each other for testing. In the embodiment provided in the present application, after the data of each layer of model physical tables is generated by scheduling through the data warehouse to be tested, the spliced ​​test scripts are executed for the data of each layer of model tables to generate the test results of the corresponding test points.

[0104] In addition, if the above test results are to be displayed, the above results can also be displayed through a display module. Accordingly, the method further includes:

[0105] The generated test results are input into the result display module, so that the result display module displays the test results.

[0106] In actual applications, the result display module has the function of receiving and generating test results, and can also convert the generated test results and display them in other forms. For example, if the received test result fails, the result display module can display the test result through a graphic ×; if the received test result passes, the result display module can display the test result through a graphic √. The corresponding result display module can also have other display forms, such as text, action, sound, etc.

[0107] In summary, after obtaining the data warehouse model to be tested corresponding to the data warehouse to be tested, the present application obtains the metadata information of the data warehouse to be tested and the blood relationship information of the data warehouse to be tested according to the data warehouse model to be tested. Then the test data is input into the data warehouse to be tested, so that the data warehouse to be tested is scheduled and generates data for each layer of the model table. Afterwards, a test script is spliced ​​together according to the metadata information and / or blood relationship information, so that each test point of the data warehouse to be tested has a corresponding test script for testing, and finally the test script is executed for the data of each layer of the model table to generate the test results. In this way, each test point of the data warehouse has a corresponding test script for automatic testing, which improves the test efficiency of the comprehensive test of the data warehouse.

[0108] This embodiment discloses a specific application scenario, see Figure 3 As shown, the process may include the following steps:

[0109] S201: Obtain the data warehouse model to be tested corresponding to the data warehouse to be tested.

[0110] In practical applications, there is a corresponding relationship between a data warehouse and a data warehouse model. In the embodiment provided in the present application, if you want to implement a test on a data warehouse to be tested, you need to first obtain the data warehouse model to be tested corresponding to the data warehouse to be tested.

[0111] S202: According to the data warehouse model to be tested, metadata information of the data warehouse to be tested is obtained.

[0112] In practical applications, based on the correspondence between the data warehouse model to be tested and the data warehouse to be tested, the model tables of each layer of the data warehouse to be tested are obtained according to the data warehouse model to be tested, and then metadata information is obtained from the model tables of each layer of the data warehouse to be tested.

[0113] S203: Acquire the blood relationship information of the data warehouse to be tested according to the data warehouse model to be tested.

[0114] In practical applications, based on the correspondence between the data warehouse model to be tested and the data warehouse to be tested, the blood relationship information in the data warehouse to be tested is obtained according to the data warehouse model to be tested.

[0115] S204: constructing test data according to the metadata information; inputting the test data into the data warehouse to be tested, scheduling the data warehouse to be tested according to the configuration information and generating data for each layer model table.

[0116] In actual applications, specific test data is first constructed based on the obtained metadata information, and then the test data is input into the data warehouse to be tested. At this time, the data warehouse will schedule the test data according to the configuration information to generate more specific data for each layer of model tables.

[0117] S205: Using the configuration information to assemble a test script according to the metadata information and / or the blood relationship information, so that each test point of the data warehouse to be tested has a corresponding test script for testing.

[0118] In practical applications, configuration information including time and other information is used to assemble a test script according to the metadata information and / or the blood relationship information, so that each test point of the data warehouse to be tested has a corresponding test script for testing.

[0119] S206: Execute the test script for the data in the model tables of each layer to generate test results.

[0120] In actual applications, after the data of the physical tables of each layer of the model are generated by scheduling through the data warehouse to be tested, the spliced ​​test scripts are executed for the data of the model tables of each layer to generate the test results of the corresponding test points.

[0121] S207: Output the generated test result.

[0122] In actual application, the generated test results are input into the result display module, and finally the result display module outputs and displays the test results.

[0123] In summary, after obtaining the data warehouse model to be tested corresponding to the data warehouse to be tested, the present application obtains the metadata information of the data warehouse to be tested and the blood relationship information of the data warehouse to be tested according to the data warehouse model to be tested. Then the test data is input into the data warehouse to be tested, so that the data warehouse to be tested is scheduled and generates data for each layer of the model table. Afterwards, a test script is spliced ​​together according to the metadata information and / or blood relationship information, so that each test point of the data warehouse to be tested has a corresponding test script for testing, and finally the test script is executed for the data of each layer of the model table to generate the test results. In this way, each test point of the data warehouse has a corresponding test script for automatic testing, which improves the test efficiency of the comprehensive test of the data warehouse.

[0124] Based on the data warehouse testing method provided in the above embodiment, the present application also provides a data warehouse testing device. The data warehouse testing device is described below in conjunction with the embodiments and drawings.

[0125] Figure 4 A structural diagram of a data warehouse testing device provided by the present application. Figure 4 The data warehouse testing device 300 provided in the embodiment of the present application may include:

[0126] The acquisition module 301 is used to acquire the data warehouse model to be tested corresponding to the data warehouse to be tested;

[0127] The metadata information module 302 is used to obtain metadata information of the data warehouse to be tested according to the data warehouse model to be tested;

[0128] The blood relationship information module 303 is used to obtain the blood relationship information of the data warehouse to be tested according to the data warehouse model to be tested;

[0129] The data preparation module 304 is used to input the test data into the data warehouse to be tested, so that the data warehouse to be tested performs scheduling and generates data for each layer model table;

[0130] The automated testing module 305 is used to assemble a test script based on the metadata information and / or the blood relationship information, so that each test point in the data warehouse to be tested has a corresponding test script for testing; the test script is executed for the data in the model tables of each layer to generate test results.

[0131] As an implementation manner, in view of different ways of obtaining metadata information, the metadata information module 302 is specifically used for:

[0132] According to the data warehouse model to be tested, obtaining each model table of each layer of the data warehouse to be tested;

[0133] Obtain metadata information of each model table of each layer of the data warehouse to be tested.

[0134] As an implementation method, since the objects corresponding to the blood relationship are different, the blood relationship obtained may also be different. The blood relationship information module 303 is specifically used for:

[0135] According to the data warehouse model to be tested, obtaining each model table of each layer of the data warehouse to be tested;

[0136] Obtain the blood relationship between each model table of each layer of the data warehouse to be tested.

[0137] As an implementation method, different test data acquisition methods are used to obtain different test data. In addition, how the data warehouse generates data in each layer of model tables is also different. The above data preparation module 304 is also specifically used to:

[0138] constructing test data according to the metadata information;

[0139] The step of inputting the test data into the data warehouse to be tested, so that the data warehouse to be tested performs scheduling and generates data of each layer model table, includes:

[0140] The test data is input into the data warehouse to be tested, and the data warehouse to be tested is scheduled according to the configuration information to generate data of each layer model table.

[0141] As another implementation, for acquiring test data from production bleaching data and generating data of each layer model table according to configuration information, the data preparation module 304 is further specifically used for:

[0142] Acquire production bleaching data, and use the production bleaching data as test data;

[0143] The step of inputting the test data into the data warehouse to be tested, scheduling and generating data of the physical tables of each layer model according to the data warehouse to be tested includes:

[0144] The test data is input into the data warehouse to be tested, and the data warehouse to be tested is scheduled according to the configuration information to generate data of each layer model table.

[0145] As an implementation manner, in view of different ways of splicing the test scripts according to the metadata information and / or the blood relationship information, the above-mentioned automatic test module 205 specifically includes:

[0146] A splicing module is used to splice a test script according to the metadata information and / or the blood relationship information using the configuration information, so that each test point of the data warehouse to be tested has a corresponding test script for testing.

[0147] The execution module is used to execute the test script for the data in the model tables of each layer and generate test results.

[0148] As an implementation method, for testing many test points in the test warehouse, and the test scripts required for these test points are different, the above splicing module is specifically used to:

[0149] The configuration information is used to assemble a test script according to the metadata information, so that the metadata integrity test point, primary key uniqueness test point, primary key not empty test point, zipper table logic test point, and date field test point of the data warehouse to be tested have corresponding test scripts for testing;

[0150] Using the configuration information to assemble a test script according to the blood relationship information, so that the data warehouse record number integrity test point to be tested has a corresponding test script for testing;

[0151] The configuration information is used to assemble a test script according to the metadata information and the blood relationship information, so that the primary and foreign key relationship integrity test points of the data warehouse to be tested and the source table to target table mapping logic test points have corresponding test scripts for testing.

[0152] As another implementation, for the test point test that needs to be configured using the characteristics of a specific data warehouse project, the data warehouse test device 300 is further specifically used for:

[0153] Using the configuration information, a test script is assembled according to the standard code table designed by the model in the project and the mapping relationship between the code conversion fields in the data warehouse, so that the standard code conversion test point of the data warehouse to be tested is tested using the test script;

[0154] The configuration information is used to assemble a test script according to the clear total-to-score relationship contained in the business involved in the data warehouse to be tested, so that the total-to-score relationship continuity test point of the data warehouse to be tested is tested using the test script.

[0155] In addition, the present application also provides a data warehouse testing device, including: a memory for storing a computer program; a processor for implementing the steps of the data warehouse testing method as described in any of the above items when executing the computer program.

[0156] In summary, after obtaining the data warehouse model to be tested corresponding to the data warehouse to be tested, the present application obtains the metadata information of the data warehouse to be tested and the blood relationship information of the data warehouse to be tested according to the data warehouse model to be tested. The test data is then input into the data warehouse to be tested, so that the data warehouse to be tested is scheduled and generates data for each layer of the model table. Afterwards, a test script is spliced ​​together based on the metadata information and / or blood relationship information, so that each test point of the data warehouse to be tested has a corresponding test script for testing, and finally the test script is executed for the data of each layer of the model table to generate the test results. In this way, each test point of the data warehouse has a corresponding test script for automatic testing, thereby improving the test efficiency of the comprehensive test of the data warehouse.

[0157] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data warehouse testing method, characterized in that: The method comprises: Obtain the data warehouse model to be tested corresponding to the data warehouse to be tested; According to the data warehouse model to be tested, obtaining metadata information of the data warehouse to be tested; According to the data warehouse model to be tested, obtaining the blood relationship information of the data warehouse to be tested; Input the test data into the data warehouse to be tested, so that the data warehouse to be tested performs scheduling and generates data of each layer model table; Using the configuration information to assemble a test script according to the metadata information and / or the blood relationship information, so that each test point of the data warehouse to be tested has a corresponding test script for testing; Execute the test script for the data in the model tables of each layer to generate test results; Using the configuration information, a test script is assembled according to the standard code table designed by the model in the project and the mapping relationship between the code conversion fields in the data warehouse, so that the standard code conversion test point of the data warehouse to be tested is tested using the test script; Using the configuration information, a test script is assembled according to the clear total-to-score relationship contained in the business involved in the data warehouse to be tested, so that the continuity test point of the total-to-score relationship of the data warehouse to be tested is tested using the test script; The utilizing configuration information to splice into a test script according to the metadata information and / or the blood relationship information, so that each test point of the data warehouse to be tested has a corresponding test script for testing, includes: The configuration information is used to assemble the test script according to the metadata information, so that the metadata integrity test point, primary key uniqueness test point, primary key not empty test point, zipper table logic test point, and date field test point of the data warehouse to be tested have corresponding test scripts for testing; Using the configuration information to assemble a test script according to the blood relationship information, so that the data warehouse record number integrity test point to be tested has a corresponding test script for testing; The configuration information is used to assemble a test script according to the metadata information and the blood relationship information, so that the primary and foreign key relationship integrity test points of the data warehouse to be tested and the source table to target table mapping logic test points have corresponding test scripts for testing.

2. The method according to claim 1, characterized in that The step of obtaining metadata information of the data warehouse to be tested according to the data warehouse model to be tested includes: According to the data warehouse model to be tested, obtaining each model table of each layer of the data warehouse to be tested; Obtain metadata information of each model table of each layer of the data warehouse to be tested.

3. The method according to claim 1, characterized in that The step of obtaining the blood relationship information of the data warehouse to be tested according to the data warehouse model to be tested includes: According to the data warehouse model to be tested, obtaining each model table of each layer of the data warehouse to be tested; Obtain the blood relationship between each model table of each layer of the data warehouse to be tested.

4. The method according to claim 1, characterized in that: The method further comprises: constructing test data according to the metadata information; The step of inputting the test data into the data warehouse to be tested, so that the data warehouse to be tested performs scheduling and generates data of each layer model table, includes: The test data is input into the data warehouse to be tested, and the data warehouse to be tested is scheduled according to the configuration information to generate data of each layer model table.

5. The method according to claim 1, characterized in that: The method further comprises: Acquire production bleaching data, and use the production bleaching data as test data; The step of inputting the test data into the data warehouse to be tested, scheduling and generating data of the physical tables of each layer model according to the data warehouse to be tested includes: The test data is input into the data warehouse to be tested, and the data warehouse to be tested is scheduled according to the configuration information to generate data of each layer model table.

6. A data warehouse testing device, characterized in that: include: An acquisition module is used to acquire the data warehouse model to be tested corresponding to the data warehouse to be tested; A metadata information module, used to obtain metadata information of the data warehouse to be tested according to the data warehouse model to be tested; A blood relationship information module, used to obtain blood relationship information of the data warehouse to be tested according to the data warehouse model to be tested; A data preparation module, used for inputting test data into the data warehouse to be tested, so that the data warehouse to be tested performs scheduling and generates data for each layer model table; An automated testing module, used to use the configuration information to assemble a test script according to the metadata information and / or the blood relationship information, so that each test point of the data warehouse to be tested has a corresponding test script for testing; Execute the test script for the data in the model tables of each layer to generate test results; The automated testing module is specifically used to use the configuration information to splice the test script according to the metadata information, so that the metadata integrity test point, primary key uniqueness test point, primary key not null test point, zipper table logic test point, and date field test point of the data warehouse to be tested have corresponding test scripts for testing; Using the configuration information to assemble a test script according to the blood relationship information, so that the data warehouse record number integrity test point to be tested has a corresponding test script for testing; Using the configuration information to stitch together a test script according to the metadata information and the blood relationship information, so that the primary and foreign key relationship integrity test points of the data warehouse to be tested and the source table to target table mapping logic test points have corresponding test scripts for testing; The automated testing module is further used to use the configuration information to assemble a test script according to the standard code table designed by the model in the project and the mapping relationship between the code conversion fields in the data warehouse, so that the standard code conversion test point of the data warehouse to be tested is tested using the test script; The configuration information is used to assemble a test script according to the clear total-to-score relationship contained in the business involved in the data warehouse to be tested, so that the total-to-score relationship continuity test point of the data warehouse to be tested is tested using the test script.

7. A data warehouse testing device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the data warehouse testing method as claimed in any one of claims 1 to 5 when executing the computer program.

Images