MPP (Massively Parallel Processor) database and Hadoop cluster data intercommunication method, tool and realization method

A technology for Hadoop cluster and data intercommunication, applied in the field of data intercommunication between MPP database and Hadoop cluster, and can solve problems such as the inability of MPP database and Hadoop business to communicate with each other.

Active Publication Date: 2015-04-29
TIANJIN NANKAI UNIV GENERAL DATA TECH
View PDF4 Cites 20 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] The problem to be solved in the present invention is to propose a method and a data intercommunication tool for MPP database and

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • MPP (Massively Parallel Processor) database and Hadoop cluster data intercommunication method, tool and realization method
  • MPP (Massively Parallel Processor) database and Hadoop cluster data intercommunication method, tool and realization method
  • MPP (Massively Parallel Processor) database and Hadoop cluster data intercommunication method, tool and realization method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0070] The present invention will be described in detail below with reference to the accompanying drawings and examples. It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0071] The invention provides a data intercommunication tool and a data intercommunication method between an MPP database and a Hadoop cluster, including a method for directly performing data intercommunication between an MPP database and a Hadoop cluster by using the data intercommunication tool, and a method for performing data intercommunication through TXT transfer.

[0072] 1. If figure 2 As shown, the data is directly exported from the MPP database to the Hadoop cluster. The computing nodes of the MPP database access the data nodes of the Hadoop cluster through the data interworking tool, and directly export the data to the Hadoop cluster without transferring the data between the MPP dat...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides an MPP (Massively Parallel Processor) database and Hadoop cluster data intercommunication method, a tool and a realization method, comprising a method for intercommunicating data between an MPP database and a Hadoop cluster by utilizing a data intercommunication tool and a method for intercommunicating the data through TXT transmit; the data is directly exported (imported) into the Hadoop cluster from the MPP database, and does not need to be transferred through a storage unit except the MPP database and the Hadoop cluster; and thereby the export process is more efficient. If the data is needed to be processed secondly through the Hadoop cluster, the TXT format transmit way is selected; according to the invention, the problem that the data between the MPP databast and the Hadoop business cannot be intercommunicated can be solved; the mashup of two business platforms of the MPP database and the Hadoop cluster is realized.

Description

technical field [0001] The invention relates to the field of distributed databases, in particular to a data intercommunication method between an MPP database and a Hadoop cluster, a tool and an implementation method thereof. Background technique [0002] Before the advent of the Internet, data was mainly generated through man-machine conversations, mainly structured data. For this kind of transactional data, end users pay more attention to the addition, deletion, modification and query of data, and the corresponding data processing is called OLTP (Online Transaction Processing, online transaction processing). The traditional relational database (RDBMS) is mainly designed and developed for this requirement, and has occupied an important position in the past 30 years. During this period, the growth of data was slow, and the systems were relatively isolated. Traditional databases can basically meet various application requirements. [0003] With the emergence and rapid develo...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/25
Inventor 陈雨夏旭东崔维力武新
Owner TIANJIN NANKAI UNIV GENERAL DATA TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap