Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method and system for detecting data consanguinity of HIVE database

A database and data technology, which is applied in the field of blood relationship detection of HIVE database data, can solve the problems of high maintenance cost, common report work, etc.

Pending Publication Date: 2021-06-08
吉林亿联银行股份有限公司
View PDF4 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

With the operation of the system and the continuous adjustment of related business systems in the actual application process, more and more data nodes have problems, and the maintenance cost is high. Only a few commonly used reports work normally.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for detecting data consanguinity of HIVE database
  • Method and system for detecting data consanguinity of HIVE database
  • Method and system for detecting data consanguinity of HIVE database

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0036] Next, the technical solutions in the embodiments of the present invention will be apparent from the embodiment of the present invention, and it is clearly described, and it is understood that the described embodiments are merely embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, there are all other embodiments obtained without making creative labor without making creative labor premises.

[0037] Such as figure 1 As shown, a method for detecting a method of detecting a Hive Database Data Blocker, wherein the method can include the following steps:

[0038] S101, configure the LineAgeLogger hook function;

[0039] When you need to detect the Hive database data, you first configure the LineAgeLogger hook feature above.

[0040] S102, based on the LineAgeLogger hook function, resolve HiveSQL, generate hive.log logs;

[0041] After configuring the Lineelogger Hook function, the configuration-based LineLogger hook...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method and a system for detecting data consanguinity of an HIVE database. The method comprises the following steps of: configuring a LineageLogger Hook function; analyzing HiveSql on the basis of the LineageLogger Hook function, and generating a hive.log; performing data cleaning on the hive.log to form a JOIN format, and importing the cleaned data into an open source graph database neo4j; querying a dependency relationship between the fields by utilizing a neo4j interface; calling a neo4j API of the graph database, analyzing a JSON string, and visually displaying the blood relationship of the data. According to the method, the analysis and carding of the data blood relationship between each data table and field can be effectively completed.

Description

Technical field [0001] The present invention relates to the field of data governance technologies, in particular, to a method and system for detecting the blood of the Hive Database. Background technique [0002] Since the first year of 2013, the big data has brought new opportunities and challenges to the development of all walks of life, and the emphasis on the value of the value contained in mass data is increasing. The data warehouse is a collection of all common, important business related indicators data from massive data, reducing the time cost of data retrieval, improving data quality and consistency, improving the application of historical data, thereby better mining Data hidden value. [0003] Data blood relational image depicts data from bottom to the upper layer, accurately and clearly reveals the blood relationship between the data entities at all levels, which supports the development, testing and operation of the business system. It records the entire history of da...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/215G06F16/248G06F16/28
CPCG06F16/215G06F16/248G06F16/283
Inventor 苏瑀陈筱进刘登贺张世杰
Owner 吉林亿联银行股份有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com