A field-level pedigree analysis method and device based on etl
What is Al technical title?
Al technical title is built by PatSnap Al team. It summarizes the technical point description of the patent document.
An analysis method and field technology, applied in the field of ETL, can solve problems such as heavy workload, reduce workload, improve work efficiency, and avoid multiple analyses.
Active Publication Date: 2019-06-04
GUANGDONG KINGPOINT DATA SCI & TECH CO LTD
View PDF5 Cites 0 Cited by
Summary
Abstract
Description
Claims
Application Information
AI Technical Summary
This helps you quickly interpret patents by identifying the three key elements:
Problems solved by technology
Method used
Benefits of technology
Problems solved by technology
Existing technologies require a large amount of work for lineage analysis at the field level and require multiple analyzes
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more
Image
Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
Click on the blue label to locate the original text in one second.
Reading with bidirectional positioning of images and text.
Smart Image
Examples
Experimental program
Comparison scheme
Effect test
Embodiment 1
[0037] Such as figure 1 As shown, it is a flow chart of an ETL-based field-level lineage analysis method provided by the present invention. This ETL-based field-level lineage analysis method includes the following steps:
[0038] Step S1, perform division operation on the content of the ETL task script file, and record the division operation result in the pedigree analysis subsidiary table.
[0039] Specifically, the content of the ETL task script file is analyzed and the operation is divided, each SQL statement is regarded as an operation, and the operation is divided according to the content of the ETL task script file according to this rule. For each step of the ETL, Each complete SQL language is numbered from small to large in logical order, named according to "step name + number" to form the division operation result, and recorded in the lineage analysis subsidiary table.
[0040] Step S2, according to the needs of the user, locate a certain field in a certain step for l...
Embodiment 2
[0056] Such as figure 2 As shown, it is a functional block diagram of an ETL-based field-level lineage analysis device provided by the present invention, and a kind of ETL-based field-level lineage analysis device includes:
[0057] The division operation module 1 is used to perform division operation on the content of the ETL task script file, and record the division operation result in the pedigree analysis subsidiary table.
[0058] Specifically, the content of the ETL task script file is analyzed and the operation is divided, each SQL statement is regarded as an operation, and the operation is divided according to the content of the ETL task script file according to this rule. For each step of the ETL, Each complete SQL language is numbered from small to large in logical order, named according to "step name + number" to form the division operation result, and recorded in the lineage analysis subsidiary table.
[0059] The positioning module 2 is used to locate a certain ...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more
PUM
Login to view more
Abstract
The invention provides a blood-relationship analysis method and a blood-relationship analysis device of field level on the basis of ETL (Extract Transform Loading). The device comprises a dividing operation module, a positioning module, a positioning and searching module, a first retrieval module, a second retrieval module, a third retrieval module and a recording module. Compared with the prior art, the blood-relationship analysis method and device provided by the invention have the advantages that the displaying of the change process of the fields on the level of the fields is realized, an iterative method is adopted to carry out blood-relationship analysis on involved relevant fields in parallel by establishing a blood-relationship table and a blood-relationship subsidiary table, so that the working efficiency is improved; and blood-relationship analysis graphics of all the fields of all the steps are recorded by adopting the blood-relationship table, and when the blood-relationship of the fields is involved again later, retrieval analysis again is not needed, so that multiple analyses are avoided and the workload is reduced.
Description
technical field [0001] The invention relates to the technical field of ETL in the process of building a data warehouse, in particular to an ETL-based field-level lineage analysis method and device. Background technique [0002] As the business of the enterprise continues to expand, the amount of data is also increasing. In order to achieve better and efficient management, the enterprise needs to integrate and analyze the data of different businesses in various regions. At present, business intelligence technology is used by more enterprises to assist the company to make better decisions, reduce risks, and improve performance. Business intelligence technology generally consists of data warehouse, online analytical processing, data mining, data backup and recovery, etc. The construction of a data warehouse requires ETL processing of data, which is the most important step in ensuring data quality. [0003] ETL, including data extraction (Extract), data transformation (Transfo...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more
Application Information
Patent Timeline
Application Date:The date an application was filed.
Publication Date:The date a patent or application was officially published.
First Publication Date:The earliest publication date of a patent with the same application number.
Issue Date:Publication date of the patent grant document.
PCT Entry Date:The Entry date of PCT National Phase.
Estimated Expiry Date:The statutory expiry date of a patent right according to the Patent Law, and it is the longest term of protection that the patent right can achieve without the termination of the patent right due to other reasons(Term extension factor has been taken into account ).
Invalid Date:Actual expiry date is based on effective date or publication date of legal transaction data of invalid patent.
Login to view more 
Login to view more