Distributed file system and method for processing multiple replica data in distributed file system

A distributed file and data technology, applied in the field of data processing, can solve problems such as reading errors, failure to know data node data damage, failure to guarantee data consistency of multiple copies, etc., to achieve the effect of ensuring consistency

Active Publication Date: 2014-04-16
TENCENT CLOUD COMPUTING BEIJING CO LTD
View PDF4 Cites 19 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The copy data saved on the data node is written in advance. During the writing process, due to various reasons such as writing errors or data node failures, the data written to some data nodes is often damaged; The node server records the address sets o

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed file system and method for processing multiple replica data in distributed file system
  • Distributed file system and method for processing multiple replica data in distributed file system
  • Distributed file system and method for processing multiple replica data in distributed file system

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0022] In order to make the objectives, technical solutions and advantages of the present invention clearer, the following further describes the present invention in detail with reference to embodiments and drawings.

[0023] In the present invention, the master node server not only records the address set of multiple copies of the data block, but also records the correct version number of the multiple copy data, and judges the read copy data through the correct version number to ensure multiple copies of the data block Data consistency. See figure 2 , Is a schematic flowchart of a method for processing multiple copy data in a distributed file system of the present invention, which includes the following steps:

[0024] In step 201, the client obtains the address set and correct version number of multiple copies of the data block from the master node server.

[0025] Each file is divided into multiple data blocks, and each data block has multiple copies of data, which are stored on...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a distributed file system and a method for processing multiple replica data in the distributed file system. The method comprises the steps that a client side obtains an address set and correct version numbers of multiple replica data of data blocks from a main node server; the client side reads replica data from data nodes corresponding to all node addresses contained in the address set; the client side judges if version numbers contained in the read replica data are consistent with the correct version numbers, if yes, the read replica data are reserved, and if not, the read replica data are eliminated. By means of the distributed file system and the method for processing the replica data in the distributed file system, consistence of the multiple replica data in the distributed file system can be guaranteed.

Description

technical field [0001] The invention relates to data processing technology, in particular to a method for processing multiple copies of data by a distributed file system and the system. Background technique [0002] see figure 1 , is a schematic structural diagram of a distributed file system in the prior art, and the system includes a client, a master node server, and multiple data nodes. In practical applications, each file is divided into multiple data blocks, and each data block has multiple copy data, which are respectively stored on multiple data nodes; the copy data is the backup data of the data block. [0003] When the client needs to read the copy backup data of the data block from the data node, the client obtains the address set of multiple copy data of the data block from the master node server, and reads the copy from the data node corresponding to each node address included in the address set data. [0004] The copy data saved on the data node is written in...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
CPCG06F16/178G06F16/182
Inventor 伍海君朱会灿邓大付李锐邹永强董乘宇陈晓东刘畅赵大勇杨绍鹏阙太富王磊张书鑫张银锋
Owner TENCENT CLOUD COMPUTING BEIJING CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products