Data processing method for heterogeneous distributed storage system

A distributed storage and data processing technology, which is applied in electrical digital data processing, input/output process of data processing, transmission system, etc., can solve high download costs, different download costs, heterogeneous distributed storage system repair bandwidth and High disk I/O problem, achieve the effect of low download cost, low disk IO, good MBR and MSR points

Inactive Publication Date: 2018-09-07
SHANDONG UNIV
View PDF2 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] In reality, many distributed storage systems are heterogeneous, and the new node needs to connect d storage nodes among the remaining storage nodes, and the size of the data downloaded from the d storage nodes is different, and the d storage nodes The download cost of each node is also different. In summary, the repair bandwidth and disk I / O in the heterogeneous distributed storage system are too high, so there is a high download cost.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method for heterogeneous distributed storage system
  • Data processing method for heterogeneous distributed storage system
  • Data processing method for heterogeneous distributed storage system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0036] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0037] It should be pointed out that the following detailed description is exemplary and intended to provide further explanation to the present application. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0038] It should be noted that the terminology used here is only for describing specific implementations, and is not intended to limit the exemplary implementations according to the present application. As used herein, unless the context clearly dictates otherwise, the singular is intended to include the plural, and it should also be understood that when the terms "comprising" and / or "comprising" are used in this specification, they mean There are features, steps, operations, means, components and / or combinatio...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data processing method for a heterogeneous distributed storage system. Storage nodes in the distributed storage system are divided into two parts including a coding repairingnode set and a replica repairing node set; after being coded, original data is put in the storage codes, which are composed of multiple coding repairing nodes and replica repairing nodes; according to the maximum-flow minimum-cut theorem, the fact that various nodes satisfy the basic condition of reconstructing an original data file is determined; the downloading cost as low as possible is obtained under the limitation of the basic condition; a minimum repairing bandwidth point and a minimum storage point are obtained by calculation; and thus, reconstruction is completed.

Description

technical field [0001] The invention relates to a data processing method of a heterogeneous distributed storage system. Background technique [0002] With the development of computer networks, the amount of network information data is becoming larger and larger, and traditional file storage systems cannot meet the needs of high capacity, high reliability, and high performance. Distributed storage system because of its good scalability and high reliability. However, in a distributed storage system, the nodes that store data are unreliable. [0003] In order to be able to provide reliable storage services by unreliable storage nodes, it is necessary to introduce redundancy into the storage system. The easiest way to introduce redundancy is to directly back up the original data. Although direct backup is simple, its storage efficiency and system reliability are not high, and the method of introducing redundancy through coding can improve its storage efficiency. [0004] In t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L29/08G06F3/06
CPCG06F3/061G06F3/0629G06F3/067H04L67/06H04L67/1097
Inventor 曹叶文艾伦
Owner SHANDONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap