Distributed storage method and distributed storage device based on network coding

A distributed storage and network coding technology, applied in the field of computer storage, can solve problems such as excessive disk IO and reduced repair bandwidth

Active Publication Date: 2013-10-02
HUAZHONG UNIV OF SCI & TECH
View PDF5 Cites 37 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] The present invention provides a distributed storage method based on network coding, and provides its device at the same time, which solves the problem that the disk IO of the storage node is too large in the existing distributed stora

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed storage method and distributed storage device based on network coding
  • Distributed storage method and distributed storage device based on network coding
  • Distributed storage method and distributed storage device based on network coding

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0117] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0118] Such as figure 1 As shown, the embodiment of the present invention includes a data encoding step, a data decoding step and a data repairing step, and is applicable to a name node NS and 5 storage nodes {DS 1 ,DS 2 ,DS 3 ...DS 5} constitutes a distributed storage system, wherein n=4 storage nodes used to store file blocks are called data nodes, which are d 0 、d 1 、d 2 、d 3 ;

[0119] (1) Data encoding steps, such as figure 2 shown, including the following sub-steps:

[0120] (1.1) Data block:

[0121] Divide the original data D with a size of 64M into original data blocks D of c=k×(d+1+i-k)-(i+1)×i / 2=4 blocks of equal size g , g=0, 1...3, k=2, d=3, coding redundancy parameter i=0;

[0122] where D 0 、D 1 、D 2 、D 3 The size of the original data D is 16M; the size of the original data D is an integer multiple of c, so L B = 0;

[...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a distributed storage method and a distributed storage device based on network coding, and belongs to the technical field of computer storage. The method and the device solve the problem that disk IO (Input/Output) of storage nodes is too large in the existing distributed storage method based on the network coding. The distributed storage method is suitable for a distributed storage system, and comprises the steps of data coding, data decoding and data repair. The distributed storage device comprises a data coding module, a data decoding module and a data repair module. According to the method and the device, when a data node is damaged, data not more than the size of a primary file D is downloaded from d data nodes, and the damaged data is repaired, so that the repair band width is reduced effectively; and randomly selected gamma coded data blocks are downloaded directly from the d data nodes, and the data blocks are not subjected to linear operation in the data nodes, so that the disk IO of the storage nodes can be reduced on the premise that the high usability of the data is ensured, and the disk IO efficiency of the data nodes is improved effectively.

Description

technical field [0001] The invention belongs to the technical field of computer storage, and in particular relates to a network coding-based distributed storage method and a device thereof. Background technique [0002] With the continuous improvement of the degree of social informatization, the generation of information has shown explosive growth, and people's demand for information storage has also continued to increase. In the commercial field, the company's business data volume has doubled, including multimedia data, e-commerce data, web search data, and so on. The ever-increasing data storage has higher and higher requirements on storage technology. [0003] Compared with the traditional file system, the distributed storage system has the following advantages: scalability, reliability, high throughput and low system cost. Today, the cluster size of the distributed storage system in large Internet companies is rapidly expanding, and the storage cost is also increasing. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
Inventor 冯丹李白施展柳青焦田丰
Owner HUAZHONG UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap