Data blood relationship analysis method based on compiling mode

Abstract

The invention discloses a data blood relationship analysis method based on a compiling mode, which comprises the following steps: S1) firstly collecting SQL scripts and metadata on a data governance platform to form a uniform format; s2) analyzing the collected SQL scripts to form an abstract syntax tree, and performing semantic analysis according to metadata, or explaining and executing the SQL; s3) constructing a data blood relationship based on the SQL syntax tree with semantic information and the data blood relationship model; s4) performing position layout of the graphic elements according to the data blood relationship, and performing visual display and data traceability operation; and S5) determining a unique identifier of each data conversion in the data blood relationship, and carrying out persistence on the data blood relationship. According to the method, a correct traceability result can be obtained, good data blood relationship analysis support for numerous databases is realized, and a new data governance platform in the future is also easily supported.

Description

technical field [0001] The invention relates to a data blood relationship analysis method, in particular to a data blood relationship analysis method based on a compiling mode. Background technique [0002] With the rapid development and application of technology in the field of big data, data lineage analysis has become an important component of data governance, and data lineage analysis technology has become an important means of data traceability, data value evaluation, data compliance certification, and data quality assessment. At present, there are three main types of data lineage analysis techniques: [0003] 1. Based on pattern matching; [0004] 2. The method based on label technology; [0005] 3. The method based on code analysis technology. [0006] Based on the method of code analysis technology, by analyzing the code used for data processing, conversion, and transmission, it can completely track the flow of data, know where the data comes from, where it goes, ...

AI Technical Summary

Problems solved by technology

[0011] 1. The data lineage analysis tool is bound to a specific platform and only supports this platform

[0012] 2. For the problems found, we can only rely on the provider of the data governance platform to solve them

[0013] 3. Unable to integrate and analyze the data lineage of multiple data platforms

[0016] 2. The existing data blood relationship development tools currently lack a comprehensive and systematic description of data blood relationship, and there is no unified data blood relationship model, which is not conducive to managing data between multiple data governance platforms

[0017] 3. There is no clear algorithm for how to use data blood relationship to trace and persist data

However, the existing data blood relationship analysis tools generally use the SQL parser provided by the data governance platform itself, which cannot completely parse the SQL statements of other manufacturers, and loses cross-platform

Images