Method for data migration of non-relational database HBase

Abstract

The invention discloses a method for data migration of a non-relational database HBase. According to the method, by reading configuration file information of users, data within a time period designated by the users are migrated from an old HBase cluster to a new HBase cluster. The method has the advantages that by designing RowKey, group information and time information of the data are embodied on the RowKey, and whether the data corresponding to the RowKey are within a migration time range designated by the users can be directly identified by scanning the RowKey during data migration; groups serve as data migration units, every group serves as a check point, the range of the non-synchronous data between the old HBase cluster and the new HBase cluster can be rigidly controlled within one group when abnormal conditions occur, and time required for restoring data synchronization is reduced; log information is recorded in the data migration process to be used for treating the abnormal conditions in the data migration process, synchronization of the data between the old cluster and the new cluster is guaranteed, and data migration between the HBase clusters can be finished accurately and efficiently.

Description

technical field [0001] The invention relates to a data migration technology of a non-relational database, in particular to a data migration method of a non-relational database HBase. Background technique [0002] Database technology is one of the fastest-growing fields in computer science and technology, and it is also one of the most widely used technologies. In recent years, with the development of the Internet, especially the rise of Internet Web2.0, traditional relational databases are dealing with The ultra-large-scale and high-concurrency SNS-type websites have been unable to do what they want, but the non-relational database has developed very rapidly due to its own characteristics in dealing with large-scale data and high-concurrency access. . [0003] HBase is a distributed, column-oriented non-relational database, suitable for applications that require real-time random read and write of very large-scale data. But as time goes by, the amount of data that needs to ...

AI Technical Summary

Problems solved by technology

Since HBase is a distributed database developed on the distributed file system HDFS, most of the existing data migration solutions for HBase are based on HDFS data blocks, copying and transferring HDFS data blocks storing HBase data to new clusters , because the data block transfer of HDFS must transfer the entire block, and cannot be split for transfer, so this transfer method is not flexible enough, and the data that needs to be transferred cannot be customized by the user, and data migration can only be performed by a user with Professional knowledge of the database administrator to operate, increasing the burden on users

Images