Data backup method and system based on Hadoop distributed file system

Abstract

The invention discloses a data backup method and system based on a Hadoop distributed file system. The method comprises the steps that a file folder is backed up in a snapshot mode through an HDFS client side, a time point snapshot of the file folder is generated through the client side, and data in the file folder is stored in an external storage medium. The system comprises an HDFS system and a storage server connected with the system, wherein the storage server comprises a storage medium and a file index database; the storage medium is used for storing system file data, and the file index database is used for storing system file metadata. According to the method, the security of the data in the HDFS can be improved, the Hadoop cluster is prevented from disasters, the system data can be automatically and quickly recovered, and the integrity and consistency of company data are protected.

Description

technical field

[0001] The invention relates to a data backup method and system, in particular to a data backup method and system based on a Hadoop distributed file system. Background technique

[0002] "Big data" has existed for a long time in the fields of physics, biology, environmental ecology, military, finance, communication and other industries, but it has attracted people's attention because of the development of the Internet and information industry in recent years. With the rapid development and popularization of computer and information technology, big data has increasingly demonstrated its advantages, and the scale of industrial application systems has expanded rapidly, and the data generated by industrial applications has grown explosively.

[0003] Hadoop implements a highly fault-tolerant distributed file system (Hadoop Distributed FileSystem, HDFS), which is used to solve problems such as low-cost hardware, scalable super-large clusters, and storage and acces...

Images