Unlock instant, AI-driven research and patent intelligence for your innovation.

A method and device for etl testing

A technology of test sub and test object, applied in the field of data warehouse analysis, can solve the problems of low test efficiency, complex test process, huge amount of tasks, etc., and achieve the effect of reducing test complexity, small test scope, and clear test scope

Active Publication Date: 2016-04-20
ALIBABA GRP HLDG LTD
View PDF4 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Using the existing technology to test the test object as a whole will make the test process very complicated and the task load is huge. If there is a problem in one of the small details, the entire test result will be wrong, and the test needs to be re-tested, thus making the entire ETL Test efficiency is very low

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A method and device for etl testing
  • A method and device for etl testing
  • A method and device for etl testing

Examples

Experimental program
Comparison scheme
Effect test

test Embodiment 1

[0079]

[0080] (6) Verification of test results

[0081] Test result verification consists of two steps:

[0082] The first step is to verify whether the number of records is consistent. If the number of records is inconsistent, there must be a problem. Check the problem and find out the reason. In the second step, if the number of records is consistent, check whether the value is correct.

[0083] Specifically, two methods can be used to achieve this:

[0084] Method 1: use minus

[0085] SELECT * FROM target_a

[0086] MINUS

[0087] SELECT * FROM test_map;

[0088] It is worth noting that the positions of the two tables must be exchanged here for comparison. When the Minus function is compared, the first table shall prevail, and the inconsistency between the first table and the second table shall be found. That is: find out the results that are inconsistent with the map table in the sumdt0 table.

[0089] Method 2: Write a script for verification.

[0090] The ...

test Embodiment 2

[0134] Test Case 2: Indicator Check

[0135] Inserting results

[0136] selectid, sum(a.gmv) asgmvfromtable01agroupbya.id

[0137] Sub-step S31-2, obtaining source table information on which the current test sub-object depends. In a specific implementation, data preparation can be performed directly through SQL statements. Such as insertintotestable(a,b)values(a,b); or, you can write a stored procedure or a java program to achieve, such as: write a loop to realize the Cartesian set of certain fields, assign values, etc., and generate SQL statements to realize the data Preparation, the data prepared here is only the test data for a single test sub-object, and the structure is relatively simple.

[0138] For example, it is analyzed that the source table that test sub-object 1 depends on is table01, so the data preparation script for table01 data preparation:

[0139] insertintotable01(id, gmv) values(1, 200);

[0140] insertintotable01(id, gmv) values(2, 300);

[0141] Sub...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a method and a device for ETL (extract-transform-load) tests. The method includes: dividing ETL test objects into test subobjects according to preset rules; and testing the test subobjects one by one. Complexity of ETL test objects can be reduced, and test efficiency can be improved.

Description

technical field [0001] The present application relates to the technical field of data warehouse analysis, in particular to a method and device for ETL testing. Background technique [0002] The data warehouse is an independent data environment, which needs to import data from the online transaction processing environment, external data sources and offline data storage media into the data warehouse through the extraction process. Its purpose is to establish a structured data storage space, separate data from different data sources, form a unified and effective data set, and finally process and integrate it into the required data. [0003] ETL is the process of data extraction (Extract), cleaning (Cleaning), transformation (Transform), and loading (Load). It is an important part of building a data warehouse. The user extracts the required data from the data source, after data cleaning, and finally loads the data into the data warehouse according to the pre-defined data wareho...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
Inventor 孟祥敏
Owner ALIBABA GRP HLDG LTD