Metadata-based data blood relationship analysis method and system

A blood relationship and analysis method technology, applied in the field of data processing, can solve problems such as inaccurate relationships, manual maintenance omissions, time-consuming and labor-intensive problems

Inactive Publication Date: 2019-12-10
BEIJING SOHU NEW MEDIA INFORMATION TECH
View PDF5 Cites 18 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] 2. The fields of the basic data table need to be modified for some reason, and its impact on the data warehouse needs to be evaluated, which is time-consuming and laborious, and then the plan is made
[0006] At present, we can only rely on manual maintenance. Once the script changes, manual maintenance is missed or not timely, which will cause inaccurate relationships

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Metadata-based data blood relationship analysis method and system
  • Metadata-based data blood relationship analysis method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0040] Such as figure 1 As shown, it is a method flowchart of Embodiment 1 of a metadata-based data blood relationship analysis method disclosed in the present invention, and the method may include the following steps:

[0041] S101. Define the lexical rules and grammatical rules of the structured query language through an open source syntax analyzer, and analyze the lexical rules and the grammatical rules of the structured query language, and convert the stru...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a metadata-based data blood relationship analysis method and system, and the method comprises the steps: defining a lexical rule and a grammatical rule of a structured query language through an open-source grammatical analyzer, analyzing the lexical rule and the grammatical rule of the structured query language, and converting the structured query language into an abstractsyntax tree; traversing the abstract syntax tree, and abstracting a basic composition unit of query; traversing the basic composition units of the query to generate an execution operation tree; executing operation tree transformation through a logic layer optimizer; traversing the execution operation tree and translating the execution operation tree into a task tree; transforming the task tree through a physical layer optimizer to generate a final execution plan; analyzing the statement of the Hive query language based on the final execution plan, and analyzing an input/output table, fields and corresponding processing conditions. The relationship between each data table and each field can be effectively sorted, and the blood relationship of data can be analyzed.

Description

technical field [0001] The present invention relates to the technical field of data processing, in particular to a metadata-based blood relationship analysis method and system. Background technique [0002] Simply put, the blood relationship of data is the relationship between the upstream and downstream sources and destinations of data, and the data input source and output source. The important role of data blood relationship is self-evident. For example, if there is a problem with a piece of data, you can check upstream based on the blood relationship to see where the problem is. In addition, the dependency relationship between the tasks that produce these data can also be established through the blood relationship of the data, and then assist the work scheduling of the scheduling system, or be used to judge which downstream data a failed or wrong task may affect, and so on. As the number of tables connected to the data warehouse and the number of models established incre...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/242G06F16/22G06F16/28
CPCG06F16/2433G06F16/2246G06F16/284
Inventor 郑波张强饶鑫淞杨川明
Owner BEIJING SOHU NEW MEDIA INFORMATION TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap