Data re-deletion method and storage equipment

Abstract

The invention provides a data re-deletion method and storage equipment. The method is applied to storage equipment in a distributed storage system. The method comprises the following steps of: responding to-be-written data in an IO request and calculating a feature value of the to-be-written data; when the feature value of the to-be-written data is sent to another storage equipment according to a feature value distribution strategy, receiving a response message returned by the another storage equipment through carrying out search on the basis of the feature value of the to-be-written data; and carrying out re-deletion processing on the to-be-written data according to the response message. The method is capable of reducing the network load of the distributed storage system and reducing the probability that IO reading needs to carry out data transmission through a network; and the transmission load of a data network is reduced, so that the delay of IO reading and writing is shortened and the IO processing speed and throughput capacity are enhanced.

Description

technical field [0001] This application relates to the storage field, and in particular to a data deduplication technology. Background technique [0002] Deduplication (full name deduplication) technology is a storage technology that automatically searches for duplicate data and keeps a unique copy of the same data. Through deduplication processing, redundant data in the storage system can be eliminated and the demand for storage capacity can be reduced. [0003] There are many ways to implement deduplication technology, but the current popular online deduplication technology is based on the Hash (translated into hash or hash) algorithm to achieve deduplication. This deduplication method mainly includes: dividing the written data into For fixed-length or variable-length data blocks, the hash value (also called fingerprint) of each data block is calculated, and new data is retained and duplicate data is deleted through hash value matching. [0004] In order to improve data ...

AI Technical Summary

Problems solved by technology

[0011] In the current deduplication technology based on the distributed storage system, the deduplication copy library is distributed and saved in the corresponding storage controller according to the fingerprint mapping table. Once the fingerprint of the data in the write IO request received by the LUN of the storage controller is not in the In the fingerprint mapping table of the storage controller, the data needs to be transmitted to the storage controller where the corresponding fingerprint is located for deduplication processing. Since the data needs to be transmitted between storage controllers, this will bring a burden to the network transmission. When deleting, the throughput of reading and writing IO is limited by the network, which will also increase the delay of reading and writing IO. When the IO concurrency pressure increases, IO accumulation may cause network congestion, and the delay is unpredictable.

Images