Strip merging method and system based on erasure codes
A technology of erasure codes and stripes, which is applied in the field of erasure code-based stripe merging methods and systems, can solve problems such as large bandwidth overhead, and achieve the effect of small computing overhead and merging overhead, and reduce computing overhead
Pending Publication Date: 2021-11-09
HUAZHONG UNIV OF SCI & TECH
View PDF0 Cites 1 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0053] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.
[0054] Aiming at the defects and improvement needs of the prior art, the present invention provides a method and system for merging slices based on erasure codes, the purpose of which is to minimize the generation bandwidth of large slices. The algorithm includes the calculation method of the merge cost of two small stripes, the StripeMerge-G algorithm and the StripeMerge-P algorithm; the merge cost of two small stripes is one of the important indicators in the process of generating a large stripe. Based on this indicator, execute StripeMerge-G and StripeMerge-P algorithms generate large stripes wi...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention provides a stripe merging method and a system based on erasure codes. The method comprises the following steps: firstly determining the alignment information of erasure code stripe verification blocks based on a preprocessed hash table through a Stripe Merge-P algorithm, selecting to-be-merged erasure code stripe pairs through a greedy strategy for merging, and directly searching the position information of the verification blocks through the hash table. The calculation overhead does not need to be additionally increased, and the overhead for selecting and combining the erasure code strip pairs is further reduced by combining a greedy strategy. Compared with a mode of directly adopting the Stripe Merge-G algorithm to calculate the merging overhead of any two erasure code strip pairs in pairs, the method has the advantages that the erasure code strip pairs with relatively low merging overhead are selected with relatively low calculation overhead through the Stripe Merge-P algorithm, and then the merged erasure code strip pairs are selected from the remaining erasure code strips by adopting the Stripe Merge-G algorithm; The overall calculation overhead and merging overhead are ensured to be relatively small, and the calculation overhead in the erasure code strip merging process is reduced.
Description
technical field [0001] The invention belongs to the field of computer storage, and more specifically, relates to a stripe merging method and system based on erasure codes. Background technique [0002] Erasure codes have been widely deployed in modern large-scale storage systems to achieve fault tolerance for efficient storage by storing data blocks and check blocks. There are many types of erasure codes, and the Reed-Solonon code is the most popular and has been widely used in the industry. In practice, large-scale storage systems store multiple stripes that are independently encoded and distributed across different sets of k+m nodes. Also, each block is usually configured with a large size, eg, 64MiB or 256MiB, to alleviate I / O overhead. Recently, enterprises have explored the concept of large stripes to suppress the proportion of parity blocks in each stripe to achieve great storage savings. However, how to generate large stripes with less overhead is still an importan...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/10
CPCG06F11/1004H03M13/373H03M13/3761H03M13/1515G06F3/0619G06F3/0644G06F3/0647G06F3/0652G06F3/0673G06F11/1076H03M13/154
Inventor 胡燏翀姚乔日程良锋张亚哲冯丹
Owner HUAZHONG UNIV OF SCI & TECH