Self-adaptation data storage and reconstruction method for coding redundancy storage system

A redundant storage and data storage technology, applied in the information field, can solve problems such as large maintenance bandwidth, data performance impact, complex management strategies, etc., and achieve the effects of reducing impact, reducing network bandwidth pressure and computing pressure, and reducing impact

Inactive Publication Date: 2014-07-09
CHENGDU INST OF BIOLOGY CHINESE ACAD OF S
View PDF3 Cites 21 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, compared with the replication redundancy strategy, the erasure code redundancy strategy will occupy more network bandwidth resources when restoring files, which will put greater pressure on the already tight network bandwidth resources in the data center. In turn, it has a greater performance

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Self-adaptation data storage and reconstruction method for coding redundancy storage system
  • Self-adaptation data storage and reconstruction method for coding redundancy storage system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0057] In this embodiment, the coding redundancy storage system is constructed by using a common PC, therefore, the system handles node failure as a normal state. When using erasure coding to reconstruct lost file blocks, according to the principle of erasure coding, the system needs at least any m data blocks, that is, m file blocks and verification data blocks to participate in the operation to restore the lost file blocks block, so the accompanying In Cast problem based on the internal network (that is, because the reconstruction node needs to call multiple file blocks, therefore, there will be multiple file blocks converging to the reconstruction node at the same time, when the network card performance of the aggregation node When it is poor, there will be data delay) will seriously reduce the concurrent transmission capacity of the cluster internal network. However, the erasure coding storage method based on multiple error correction provides more optional recovery strate...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a self-adaptation data storage and reconstruction method for a coding redundancy storage system. The self-adaptation data storage and reconstruction method includes the following steps: (1) a client terminal calculates a Hash value of a file to be stored and uploads the Hash value to a server terminal; (2) the Hash value of the file to be stored is compared with Hash values of files already stored in the server terminal; (3) if the Hash value equal to the Hash value of the file exists, the server terminal does not accept uploading of the file to be stored, if the Hash value equal to the Hash value of the file does not exist, the server terminal accepts uploading of the file to be stored, the uploaded file is partitioned, Hash values of the file partitions are calculated and stored, and the file partitions are encoded to generate verification data partitions. Compared with the prior art, the self-adaptation data storage and reconstruction method has the advantages that the Hash value of the file to be stored and the Hash values of the file partitions are recorded, the corresponding file storage and reconstruction method is selected according to conditions of the system and the client terminal, and therefore network bandwidth pressure and calculation pressure, caused by data reconstruction, on a data center are reduced.

Description

technical field [0001] The invention relates to the field of information technology, in particular to an adaptive data storage and lost data reconstruction method of a data storage system using a coding redundancy strategy as a basic storage architecture. Background technique [0002] Compared with replication redundancy technology, reliability technology based on coding redundancy has lower data redundancy and storage overhead under the premise of the same fault tolerance. However, when a node is damaged or a data block is lost in the system, the storage strategy based on replication redundancy only needs to download the same amount of data as the lost data to realize the repair process, but based on coding redundancy, such as the redundancy of erasure code The strategy needs to download at least k times the amount of lost data to decode and reconstruct the lost data. Therefore, compared with the replication redundancy strategy, the erasure code redundancy strategy will oc...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L29/08
Inventor 蒋海波李娜周星梅陈建中王晓京
Owner CHENGDU INST OF BIOLOGY CHINESE ACAD OF S
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap