Distributed file system and method for processing multiple replica data in distributed file system

Abstract

The invention discloses a distributed file system and a method for processing multiple replica data in the distributed file system. The method comprises the steps that a client side obtains an address set and correct version numbers of multiple replica data of data blocks from a main node server; the client side reads replica data from data nodes corresponding to all node addresses contained in the address set; the client side judges if version numbers contained in the read replica data are consistent with the correct version numbers, if yes, the read replica data are reserved, and if not, the read replica data are eliminated. By means of the distributed file system and the method for processing the replica data in the distributed file system, consistence of the multiple replica data in the distributed file system can be guaranteed.

Description

technical field [0001] The invention relates to data processing technology, in particular to a method for processing multiple copies of data by a distributed file system and the system. Background technique [0002] see figure 1 , is a schematic structural diagram of a distributed file system in the prior art, and the system includes a client, a master node server, and multiple data nodes. In practical applications, each file is divided into multiple data blocks, and each data block has multiple copy data, which are respectively stored on multiple data nodes; the copy data is the backup data of the data block. [0003] When the client needs to read the copy backup data of the data block from the data node, the client obtains the address set of multiple copy data of the data block from the master node server, and reads the copy from the data node corresponding to each node address included in the address set data. [0004] The copy data saved on the data node is written in...

AI Technical Summary

Problems solved by technology

[0004] The copy data saved on the data node is written in advance. During the writing process, due to various reasons such as writing errors or data node failures, the data written to some data nodes is often damaged; The node server records the address sets of multiple copies of data, and cannot know which node addresses correspond to the data on the data node is damaged; in this way, when reading multiple copies of data, the wrong data is read, and multiple copies cannot be guaranteed. Consistency of replica data

Images