Data synchronization method and device

Abstract

The invention provides a data synchronizing method and device. The data synchronizing method for an HADOOP server comprises the steps of obtaining a HDFS file directory of synchronous data in the HADOOP server; synchronizing the synchronous data under the HDFS file directory into an NFS shared directory in a third party server, wherein the NFS shared directory has a one-to-one mapping relationship with an external table in an Oracle server so as to achieve synchronization of the synchronous data from the HADOOP server to the Oracle server. According to the data synchronizing method and device, the data in the HADOOP server can be easily and quickly synchronized into the Oracle server, and therefore the efficiency of data synchronization is improved.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a data synchronization method and a data synchronization device. Background technique [0002] The current big data platform data is generally stored on the HADOOP (Haydup, a distributed system infrastructure) server. Some business systems urgently need to extract data from the HADOOP server and synchronize it to the traditional relational database, such as Oracle (a Relational database management system) server, and currently there are two main schemes for data synchronization: [0003] Solution 1: Use SQOOP open source software (an open source tool, mainly used for data transfer between HADOOP and traditional data blocks), write SQOOP synchronization script to collect data from HADOOP server to Oracle server. [0004] Solution 2: Write a JAVA (object-oriented programming language) program to query data from HIVE (a data warehouse tool based on HADOOP) interface with ...

AI Technical Summary

Problems solved by technology

[0011] In this technical solution, the purpose of simply and quickly synchronizing the data in the HADOOP server to the Oracle server is achieved by mounting the NFS (NetworkFile System, Network File System) shared directory between the HADOOP server and the Oracle server. To obtain the HDFS file directory of the synchronization data in the HADOOP server that needs to be synchronized to the Oracle server, and synchronize the synchronization data in the HDFS file directory to the NFS shared directory, and then communicate between the NFS shared directory and the external table in the Oracle server The one-to-one correspondence relationship enables the Oracle server to access the synchronized data in the NFS shared directory by reading the external table, that is, to realize the access to the synchronized data in the HDFS file directory in the HADOOP server, and then realize the data between the HADOOP server and the Oracle server Synchronization, in this way, by using NFS to allow HADOOP server and Oracle server to share resources through the network, that is, NFS allows different hardware and operating systems to share with each other through a set of RPC (Remote Procedure Call Protocol, remote procedure call protocol) The same data, as well as accessing the synchronized data in the HADOOP server through external tables, does not need to copy a large amount of data from the HADOOP server to the Oracle server for storage, thus solving the problem of complex and inefficient data synchronization process

Images