Rapid data recovery method and system based on crossed code correction and deletion

A criss-cross and erasure coding technology, which is applied in the field of fast data recovery, can solve problems such as disk failure and node failure, and achieve the effects of preventing loss, improving overall bandwidth, and reducing network pressure

Active Publication Date: 2017-06-13
INST OF COMPUTING TECH CHINESE ACAD OF SCI +1
View PDF2 Cites 25 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0026] 3) The recovery efficiency of the entire node and entire disk of the system. System failures are generally disk failures, node failures, etc. Such failures bring greater recovery challenges to the system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Rapid data recovery method and system based on crossed code correction and deletion
  • Rapid data recovery method and system based on crossed code correction and deletion
  • Rapid data recovery method and system based on crossed code correction and deletion

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0054] The purpose of the present invention is to solve the problem of inefficient data recovery in the erasure code file system. The following are the overall steps of the present invention, as follows:

[0055] (1) Adaptive multi-priority queue calculation module based on feedback mechanism

[0056] Through testing, it is found that the erasure code computing performance of the Intel ISA-L acceleration library can reach 4GB / s / core computing power. Through analysis, it is found that the Intel ISA-L acceleration library is based on SSE, AVX, and AVX2 instructions. The acceleration of SIMD instructions can complete the calculation of one strip at a time, which is 4-5 times faster than other erasure code calculation libraries in a single thread. Therefore, the emergence of intel ISA-L acceleration library makes the calculation of erasure code no longer a Therefore, based on the ISA-L acceleration library, we built an adaptive multi-priority erasure code calculation module based ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a rapid data recovery method and system based on crossed code correction and deletion. The method comprises the steps that calculation is performed according to an LRC encoding mode when data is written in, global encoding blocks are grouped in pairs, the global encoding blocks in each group are divided into two parts, after being subjected to exclusive-or operation, the second half part of one global encoding block and the first half part of the other global encoding block are written into the second half part of the current global encoding block, when the first global encoding block is lost, the second half part of the data block is read, the two second half parts, obtained before the exclusive-or operation is performed, of the lost data block are obtained, the first half part of the lost data block is obtained after the second half part, obtained after exclusive-or operation is performed, of the other global encoding block is subjected to exclusive-or operation, the data obtained before exclusive-or operation of the lost data block and the first half part of the other global encoding block are subjected to exclusive-or operation to obtain the second half part of the lost data block; the data is brushed back to a corresponding magnetic disk in a stripe mode to be stored, and after the data is written into a storage server, asynchronization longitudinal encoding calculation is performed.

Description

technical field [0001] The present invention relates to the technical field of data storage and recovery, in particular to a fast data recovery method and system based on cross erasure coding. Background technique [0002] In the context of cloud computing virtualization, the storage of massive data still exists as a key issue. Although the price of storage devices has gradually decreased, the surge in data volume still brings a large cost to the storage of data centers. Under the current situation where traditional mechanical disks and NVMe devices are parallel, mechanical disks are still the main force of storage, and NVMe devices also need some technical solutions to reduce the unit cost of storage. [0003] Erasure codes came into being in order to solve the problem that the RAID mechanism can only tolerate the loss of 2 pieces of data and the copy needs 3 times the storage. At present, erasure codes have been adopted by major open source and commercial systems. The exis...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/14G06F11/10
CPCG06F11/1004G06F11/1464
Inventor 魏征窦禹邢晶霍志刚王田
Owner INST OF COMPUTING TECH CHINESE ACAD OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap