Data blood relationship analysis method and device, storage medium and electronic equipment

CN116010461BActive Publication Date: 2026-06-23CHINA TELECOM CORP LTD
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
CHINA TELECOM CORP LTD
Filing Date
2022-12-27
Publication Date
2026-06-23

AI Technical Summary

Technical Problem

Existing data lineage generation methods can only be based on SQL parsing and cannot handle data lineage relationships in Spark and MapReduce programs.

Method used

By determining the input and output path information, adding tags, obtaining metadata, and determining the lineage relationships between libraries and tables based on the tags and metadata, the complete lineage relationships of tables can be parsed out. This method is applicable to Spark and MapReduce programs.

Benefits of technology

It implements the parsing of data lineage in Spark and MapReduce programs, accurately obtains the lineage of database tables, and supports data ETL processing in big data platforms.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116010461B_ABST
    Figure CN116010461B_ABST
Patent Text Reader

Abstract

The present disclosure provides a data blood relationship analysis method and device, a storage medium and an electronic device, and relates to the technical field of data processing. The method comprises the following steps: determining a first library and a first table according to input path information, and adding a first label to the first library and the first table; determining a second library and a second table according to output path information, and adding a second label to the second library and the second table; determining the blood relationship between the first library and the second library and the blood relationship between the first table and the second table according to the first label and the second label; obtaining metadata; and determining a data blood relationship according to the metadata and the blood relationship. The present disclosure can analyze the complete blood relationship of a table according to task information and logs.
Need to check novelty before this filing date? Find Prior Art