System and method for data migration

Abstract

The invention provides a system and a method for data migration. The system specifically comprises a configuration reading module, a reading module, a data exchange buffer module and a writing module, wherein the configuration reading module is used for reading a configuration file of a data migration task, and the configuration file comprises data reading information and data writing information; the reading module is used for reading the data according to the data reading information in the configuration file and converting the read data into an intermediate format to be written into the data exchange buffer module; the data exchange buffer module is used for temporarily storing the written intermediate format data; and the writing module is used for reading the intermediate format data from the data exchange buffer module, writing the information according to the data in the configuration file and converting the intermediate format data into a target format to be written into a target address. By the system, the data migration performance can be improved, and the maintenance cost and the development cost which are spent on data migration are lowered.

Description

technical field [0001] The present application relates to the technical field of computer system data processing, in particular to a data migration system and method. Background technique [0002] At present, with the development of network technology and the increase of information volume, a huge number of distributed heterogeneous information sources have been formed on the network. Due to the different geographical locations or the increase of new needs, various enterprises and institutions have also formed a distributed, heterogeneous and huge data collection. A very important reason for the emergence of heterogeneous data sources is the simultaneous existence of multiple commercial databases. [0003] There are various forms of heterogeneous data sources, which can be distinguished in terms of structure. They can include not only structured data in databases (such as relational databases), but also unstructured data in file systems, such as general text files, emails, ...

AI Technical Summary

Problems solved by technology

[0005] 1. Both data export and transcoding process require disk dump, because disk dump requires a lot of disk I/O (input/output, input/output), and disk is a slow device, which leads to the process of data migration It is relatively long, and the performance of data migration is low; in addition, in the long data migration process, users not only need to connect the upstream and downstream processes, monitor each process, but also monitor the disk space, which greatly increases the operation and maintenance cost;

[0006] 2. Although there are many mature dat...

Images