Metadata management method and device of HDFS storage system

Abstract

The invention discloses a metadata management method of an HDFS storage system, which is applied to a first management node of the HDFS storage system, wherein the HDFS storage system comprises a plurality of management nodes sharing a same metadata information pool, and the first management node is any one of a plurality of management nodes included in the HDFS storage system; the method comprises the following steps of receiving a service request of a client aiming at the target metadata, and responding the service request based on the metadata information pool. According to the technical scheme provided by the embodiment of the invention, a plurality of management nodes share the same metadata information pool so that the problem of inconsistent data can be avoided, and a plurality of management nodes simultaneously provide services to the outside so that the problem that a single management node is overloaded can be avoided; thus, the HDFS storage system can be in a high availability state for a long time, and the storage performance is improved. The invention further discloses a metadata management device of the HDFS storage system, and the metadata management device has corresponding technical effects.

Description

technical field [0001] The invention relates to the technical field of computer storage, in particular to a metadata management method and device of an HDFS storage system. Background technique [0002] With the rapid development of computer technology, big data applications are becoming more and more extensive, and the performance requirements for storage are getting higher and higher. HDFS (Hadoop Distributed File System, Hadoop Distributed File System) is a distributed unified storage system, which is the storage component of Hadoop for big data processing. [0003] At present, in the HDFS storage system, two management nodes are mostly configured based on the master-slave mode, that is, the master management node and the slave management node. The management node can also be called the NameNode. At the same time, the main management node is responsible for the data storage service of big data, its operation process is recorded in the log, and the log is exported accordi...

AI Technical Summary

Problems solved by technology

[0004] This method has certain disadvantages, because the logs are exported according to a certain time interval, the operation process for a short period of time before the main management node has a problem may not be recorded, there are problems of data loss and inconsistency, and, at the same time Only one management node provides external services, and it is easy to cause abnormalities in the management node due to heavy load, making the HDFS storage system unavailable and affecting storage performance

Images