An Approximate Optimal Data Fragmentation Block Rewriting Method Oriented to Data Deduplication Technology

Abstract

The invention discloses an approximately optimal data fragmentation block rewriting method oriented to data deduplication technology. With the advent of the data age, the total amount of data has shown explosive growth, and the data storage and management requirements of the information world have reached PB or even EB levels. IDC research found that nearly 75% of data in the digital world is redundant, and ESG pointed out that data redundancy in backup and archiving systems exceeds 90%. Data deduplication technology can effectively identify and eliminate duplicate data and reduce the cost of data storage. However, deduplication can lead to data fragmentation, which seriously impairs data recovery performance. The present invention can accurately identify the fragmented blocks in the data block, and alleviate the degree of data fragmentation by rewriting the fragmented blocks; at the same time, the present invention can adaptively switch the rewriting strategy (optimal rewriting strategy and Aggressive rewrite strategy), thus greatly improving data recovery performance and data deduplication rate.

Description

technical field [0001] The invention relates to the technical field of data storage and data deduplication, in particular to an approximate optimal data fragment block rewriting method oriented to data deduplication technology. Background technique [0002] With the advent of the data age, the total amount of data in the world is showing an explosive growth trend. IDC research shows that by 2020, the world's annual data growth will increase 44 times from 0.8ZB in 2009 to 35ZB. With the increase of devices such as mobile devices and sensors, the growth of data is intensified. These data sources include human genes, social networks, financial analysis, environmental protection, energy exploration, video games, and medical health. More and more applications field. These data are not only huge in amount, but also complex and diverse in data structure, which brings new challenges to data storage and management, and also increases the risk of data management. How to effectively...

AI Technical Summary

Problems solved by technology

[0003] However, data deduplication technology introduces data fragmentation, which seriously damages the read performance of the data deduplication system, that is, data recovery performance.

The main method to alleviate data fragmentation is to rewrite fragmented blocks, but the current method of rewriting fragmented blocks (referred to as the rewriting method) has the disadvantages of a large deduplication rate of lost data and a small improvement in data recovery performance.

There are two main reasons for this shortcoming: (1) The current rewriting method only sorts the containers referenced by a data segment according to the container reference rate, and selects the container with the "lowest" container reference rate one-sidedly and arbitrarily. As a result, the containers selected by them are not optimal, and thus the rewritten fragment blocks are not optimal; (2) The current rewriting algorithm cannot adaptively adjust the rewriting strategy according to different workloads

Images