Erasure code optimization method for distributed storage system

A technology of distributed storage and optimization methods, applied in the field of data storage, can solve problems such as reducing data security, and achieve the effect of reducing data volume and high read and write performance

Inactive Publication Date: 2017-08-25
深圳市联云港科技有限公司
View PDF4 Cites 21 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This method can ensure that most of the write requests are written in full stripes, avoiding the step of reading the data in the less th

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Erasure code optimization method for distributed storage system
  • Erasure code optimization method for distributed storage system
  • Erasure code optimization method for distributed storage system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0024] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, so as to define the protection scope of the present invention more clearly.

[0025] Please refer to the attached Figure 1-5 , an erasure code optimization method for a distributed storage system of the present invention, the method is to use the characteristics of erasure codes to reduce the size of erasure code stripes, and small stripes can better meet full stripe writing, for The part that is not full of the stripe minimizes the data to be read and completed; the small stripe is larger than the big IO, and the small stripe divides the requested data into many stripes, and uses the memory reference technology to merge the small stripes at the logical layer to ensure The large IO is divided, and the performance is not a...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides an erasure code optimization method for a distributed storage system. The method utilizes the characteristics of an erasure code to reduce the size of a band of the erasure code. The small strip can meet full band writing and the read and completed data is minimized for an un-full band part. When the data is greater than large IO, the small band can split request data into a plurality of bands, a memory reference technology is utilized to carry out merging of the small bands at a logical layer, so that the large IO can be split and the performance cannot be affected. According to the erasure code optimization method, the band of the erasure code is minimized, so that the data volume for reading the completed data is reduced when the un-full band is written. A front segment can ensure higher read-write performance without setting a buffer pool, so that the data security is greatly increased and the memory consumption of the front segment is reduced.

Description

technical field [0001] The invention relates to the technical field of data storage, in particular to an erasure code optimization method for a distributed storage system. Background technique [0002] With the advent of the information age, the amount of global data is in an explosive growth trend. Improving the reliability of storage systems and ensuring data availability have become the research focus of enterprises. Most of the existing distributed storage systems use multi-copy technology to improve system reliability, availability, performance and scalability. But in the era of big data, the storage scale is getting bigger and bigger, and the system overhead of multi-copy technology will be bigger and bigger. Compared with copy technology, erasure code technology has higher storage efficiency and can reduce data traffic in the network. However, erasure codes need to consume a lot of CPU resources, and the read and write process is relatively complicated, requiring a...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): H04L29/08G06F11/10
Inventor 陈仲涛
Owner 深圳市联云港科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap