Metadata processing method and device

Abstract

The invention discloses a metadata processing method and device, and is used for realizing the reliability and the availability of a metadata write operation in a distributed storage system. The metadata processing method provided by the invention comprises the following steps: a main node obtains a metadata write request sent from an access side, wherein the metadata write request comprises the metadata which needs to be uploaded into the distributed storage system, the distributed storage system comprises 2N+1 nodes, one node is a master node, rest nodes are 2N slave nodes, and N is a non-zero natural number; the master node stores the metadata according to the metadata write request, and independently sends the metadata to the 2N slave nodes; and if the master node receives confirmation information returned after at least N slave nodes in the 2N slave nodes store the metadata, the master node sends metadata write response to the access side.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a metadata processing method and device. Background technique [0002] Metadata refers to the index information of file content or simple key-value pair (English full name: Key-Value, English abbreviation: K-V) data, which has the characteristics of occupying less storage space, a large amount of data, and high reliability and availability requirements. In order to solve distributed Regarding the reliability and availability of metadata in the storage system, the current mainstream distributed storage systems provide the following processing methods for metadata: [0003] 1. Based on the master-standby mode [0004] The distributed storage system is divided into multiple units that can be expanded in parallel, with multiple main units, and each main unit is also connected to multiple backup units. The internal metadata of each unit is resolved based on the master-stand...

AI Technical Summary

Problems solved by technology

[0010] The existing technology based on the master-standby mode has at least the following problems: the master-standby asynchronous synchronization method will have a master-standby synchronization cycle for writing data, and there will be a certain time delay in the process of writing data from the master node to the slave node. Data cannot be recovered and data reliability is not high

[0011] The existing technology based on the multi-copy mode has at least the following problems: Since the data will be written into multiple copies, only a few copies of the data will be selected for each read and write operation, making the data consistency very poor. When the write operation is very frequent When , the data versions stored by each node are very different, resulting in inconsistent data storage between nodes, so complex logic cannot be processed

Since the front end interacts directly with all nodes in the cluster, the interactive node needs to forward the received data to the master node during the write operation, that is, if the front end interacts with the slave node, the write operation can only be forwarded by the slave node to the master node. The master node, so one more node forwarding process is required. When the number of nodes exceeds 5, the write performance drops significantly

Images