Self-adaptive threshold repeated data deletion method based on greedy selection

Abstract

The invention discloses a self-adaptive threshold repeated data deletion method based on greedy selection. Along with the sharp increase of the data volume in the deduplication backup system, the matching problem of the total disk capacity of the deduplication backup system and the backup data volume becomes one of main contradictions. Therefore, a rewriting algorithm is added in data deduplication to relieve damage to a deduplication backup system caused by fragmentation. According to the method, the old container with a better effect is selected and referenced for deduplication, so that the data recovery performance of a deduplication backup system is improved while the high deduplication rate is ensured. According to the self-adaptive threshold repeated data deletion method based on greedy selection, different data segments are subjected to duplicate removal by using different thresholds according to the distribution condition of the effective reference quantity of old containers involved in each data segment, and then an old container group with the maximum effective reference quantity is selected in a greedy manner, so that the recovery performance of the duplicate removal backup system is improved while the high duplicate data deletion rate is ensured.

Description

technical field [0001] The invention relates to the technical field of data storage and data deduplication, in particular to a greedy selection-based adaptive threshold deduplication method. Background technique [0002] With the widespread use of computers in various fields of society, the amount of digital information in various industries and enterprises has grown explosively, and the demand for capacity for storing data has also increased. At the same time, the surge in data volume has also brought a series of Related issues, for example, there are more and more time points when enterprises need to quickly back up and restore data, and the expenditures for data backup storage, management and related energy consumption in enterprises are increasing, which has gradually become a major factor restricting the economic development of enterprises . Studies have found that up to 60% of the data stored in the deduplication backup system is redundant, and as time goes by, the am...

AI Technical Summary

Problems solved by technology

[0004] Existing rewrite algorithms use a fixed threshold to limit old container references, such as figure 1 As shown, when the fixed threshold is 10, the rewriting effect is different for different data segments

Data segment 1 will discard some old container references with high utilization, resulting in an unnecessary reduction in the deduplication rate

And data segment 2 will refer to some old containers with low utilization, resulting in slower recovery

Images