Eureka AIR delivers breakthrough ideas for toughest innovation challenges, trusted by R&D personnel around the world.

A Consistent Hash Storage System Construction Method Based on Stripe

A storage system and construction method technology, applied in the field of data placement mechanism, can solve problems such as inability to adapt to the data recovery function, and achieve the effects of ensuring the nature of MDS, reducing the amount of data, and improving stability

Active Publication Date: 2021-07-20
INST OF COMPUTING TECH CHINESE ACAD OF SCI
View PDF11 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Because the location of a large number of data blocks has changed, additional data migration overhead is generated during the recovery process, which cannot meet the needs of the data recovery function.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Consistent Hash Storage System Construction Method Based on Stripe
  • A Consistent Hash Storage System Construction Method Based on Stripe
  • A Consistent Hash Storage System Construction Method Based on Stripe

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0057] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0058] Basic principle of the present invention is as follows:

[0059] First, build a storage system, construct a ring-shaped hash space according to the design of the consistent hash algorithm, generate a certain number of virtual nodes, and distribute these virtual nodes to the hash space evenly, and construct the storage nodes in the storage system into A number of node groups satisfying the properties of erasure code MDS, and hashing the node groups to the hash space, establishing the mapping relationship between virtual nodes and storage node groups on the hash space; each virtua...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention provides a stripe-based consistent hash storage system construction method and corresponding data placement mechanism and node change method, organizes data blocks by stripes, organizes storage nodes by node groups, and organizes stripes placed on the node group. When nodes are organized into node groups, the number of identical nodes in each node group is not greater than the number of check blocks in the stripe, so as to ensure that the placement of data blocks meets the MDS properties of erasure codes and ensures the reliability of data storage. At the same time, the present invention uses a consistent hash algorithm to select the node group with the lowest degree of difference to replace between node groups. Through the consistent hash algorithm, only the placement position of data on some virtual nodes changes, and by selecting the node group with the lowest degree of difference The node group is used as a replacement node group. Only the nodes at the changed node position are different, and the nodes at other corresponding positions are the same. At this time, the amount of data migrated is the smallest.

Description

technical field [0001] The present invention relates to the fields of data storage and erasure codes, in particular to a method for constructing a stripe-based consistent hash storage system and a corresponding data placement mechanism. Background technique [0002] In the era of big data, the storage of massive data is facing the contradiction between storage reliability and space utilization. The erasure code storage method, which not only has high space utilization efficiency, but also ensures the reliability of data storage, is more and more used in storage systems, for example, in Google's GFS, Microsoft's Azure and Facebook's storage System and other commercial systems have applications. [0003] Erasure codes are similar to RAID. Data is grouped into stripes (Stripes). Each stripe contains N blocks of data. The data is encoded by an encoding matrix to generate N encoding blocks and M check blocks, which are collectively referred to as data blocks. The encoded block ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/22G06F11/10
CPCG06F11/10G06F16/2255
Inventor 魏征窦禹邢晶霍志刚马捷
Owner INST OF COMPUTING TECH CHINESE ACAD OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com