Erasure code updating method and system based on copy data logs

A technology of data log and update method, which is applied in the direction of electrical digital data processing, response error generation, error detection/correction, etc., can solve the problem of large update delay, achieve performance improvement, improve update efficiency, and reduce computing overhead.

Active Publication Date: 2019-09-20
INST OF COMPUTING TECH CHINESE ACAD OF SCI
View PDF9 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0027] In order to solve the above-mentioned prior art in the erasure code update process, due to the large update delay caused by the existing update mechanism, the calculation overhead, network overhead and disk seek overhead caused by the insufficient utilization of the principle of data locality, due to the log access Due to the balance of log addition and recycling efficiency caused by random overhead, this invention adds data logs based on the copy mechanism at the data block end, and proposes an erasure code update method based on copy data logs

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Erasure code updating method and system based on copy data logs
  • Erasure code updating method and system based on copy data logs
  • Erasure code updating method and system based on copy data logs

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0045] In order to make the purpose, technical solution and advantages of the present invention clearer, the method and system for updating the erasure code based on the replica data log proposed by the present invention will be further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation methods described here are only used to explain the present invention, and are not intended to limit the present invention.

[0046] On the basis of the verification log, the present invention adds a copy-based data log to form a hybrid log architecture. The updated data can be aggregated on the data log side according to the principle of time-space locality, reducing a large amount of computing overhead, network overhead, and disk access overhead. . Data logs are stored in different nodes in the form of copies. On the premise of ensuring consistency, the update delay is reduced and the update IOPS is greatly improve...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to an erasure code updating method based on copy data logsreplica data log-based erasure code updating method, which comprises the following steps of: receiving updated data, and storing the updated data to a replica data log of a data end; when the copy data log is full, obtaining a verification difference value by using the updated data and the data block, updating the corresponding data block by using the verification difference value, and forwarding the verification difference value to a verification end for storing the verification block; and storing the verification difference value to a verification log of the verification end, generating verification data through the verification difference value when the verification log is full, and updating a verification block by using the verification data. According to the erasure code updating method provided by the invention, on the premise that the consistency is ensured, the calculation overhead, the network overhead and the disk channel finding overhead generated in the updating process are reduced, the updating efficiency of the erasure codes is improved, and the performance improvement of one order of magnitude is achieved in some application scenarios compared with other updating methods.

Description

technical field [0001] The invention relates to the fields of data storage and erasure codes, in particular to a hybrid log erasure code update method based on duplicate data logs. Background technique [0002] In the era of big data, the storage of massive data is facing the contradiction between storage reliability and space utilization. The erasure code storage method not only has high space utilization efficiency, but also can ensure the reliability of data storage, and is more and more used in storage systems. Component failures often threaten the reliability of clustered storage systems built from inexpensive commodity servers. Disk arrays, replicas, and erasure coding mechanisms are used to improve data reliability. However, the performance of the disk array is relatively low, and it acts on a single machine, which cannot guarantee the node data reliability of the distributed system. The copy mechanism is simple and efficient, and is currently the most common data ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/10G06F9/50
CPCG06F9/5016G06F9/5022G06F11/1012
Inventor 魏征邢晶高艳珍孙凝晖
Owner INST OF COMPUTING TECH CHINESE ACAD OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap