Unlock instant, AI-driven research and patent intelligence for your innovation.

Data migration method and device

A technology for data to be migrated, applied in the field of communications, can solve the problems of reducing the efficiency of the data migration process, unable to import HFile files, and low overall computing efficiency, so as to improve the efficiency of data migration.

Active Publication Date: 2021-06-22
ZHEJIANG UNIVIEW TECH CO LTD
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, using the batch processing distributed computing framework of the server cluster to perform calculations on text file data, its overall calculation efficiency is much lower than that of all independent servers in the cluster, and this method will reduce the efficiency of the entire data migration process
However, if the server cluster is disassembled into separate servers for calculation, there is no guarantee that the HFile files generated by each single server are compatible, and the HFile files generated by each single server cannot be imported into the HBase table of the server cluster.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data migration method and device
  • Data migration method and device
  • Data migration method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0019] In order to make the purpose, technical solution and advantages of the present application clearer, the solution of the present application will be further described in detail below with reference to the accompanying drawings.

[0020] In order to solve the problems existing in the prior art, the present invention provides a data migration method and device.

[0021] figure 1 It shows a schematic diagram of the network environment applied by the data migration method provided by the present invention, including a server cluster 1 and a server cluster 2, wherein the server cluster 1 stores a large amount of data to be migrated. The server cluster (hereinafter referred to as the source server cluster) and the server cluster 2 (hereinafter referred to as the target server cluster) is the server cluster to which the data to be migrated in the source server cluster needs to be migrated. Both the source server cluster and the destination server cluster are server clusters co...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention provides a data migration method and device, wherein the method includes: obtaining an HBase table created by a designated server; obtaining data to be migrated, and generating a data file in a specified format according to the HBase table; wherein, The data to be migrated is the corresponding data allocated to each independent server for processing respectively after splitting all the data to be migrated according to the number of independent servers obtained after dismantling the server cluster; the data files in the specified format are migrated to In the HBase table created by the specified server. The invention solves the problem in the prior art that the HFile files generated by each single server cannot be imported into the HBase table of the server cluster when the server cluster is disassembled into individual servers for calculation, and effectively improves the data migration efficiency.

Description

technical field [0001] The present invention relates to the technical field of communications, and in particular to a data migration method and device. Background technique [0002] In order to meet the requirements for the retention and migration of historical data (especially in the case of migration from relational database to non-relational database), it is usually necessary to obtain historical data from the original database and save the data according to the design requirements of the new database to the new datasheet. [0003] When the amount of data is huge, the historical data is generally exported and saved in a text file through the data export tool, and the high-performance batch processing distributed computing framework of the server cluster is used to generate the HFile file of the HBase table from the text file data, and finally the HFile The file is imported into the specified HBase table. [0004] However, using the batch processing distributed computing...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/21
CPCG06F16/214
Inventor 黄群
Owner ZHEJIANG UNIVIEW TECH CO LTD