Weak fingerprint repeated data deletion mechanism based on flash memory solid-state disk

Abstract

The present invention relates to a weak fingerprint repeated data deletion mechanism based on a flash memory solid-state disk. Oriented to online applications with repeated data, the mechanism implements a deduplication system in a solid-state disk by using hardware resources, and deduplication is performed through weak fingerprint calculation and byte direct comparison, so that the defects of the conventional deduplication technology and pre-hash deduplication technology are overcome, a higher deduplication throughput and deduplication rate are achieved, and overheads on the aspects of time, space, hardware resources and the like are minimized. Moreover, due to a low hardware overhead, parallelism in the solid-state disk can be fully used, and a plurality of deduplication parts perform deduplication in parallel, so that a write delay is greatly reduced. The mechanism is capable of combining with a flash memory solid-state disk properly, is capable of effectively eliminating repeated data, and improves the service life and performance of the solid-state disk.

Description

technical field [0001] The present invention is applicable to the technical field of data deduplication, and provides a weak fingerprint deduplication mechanism based on a flash memory solid state disk. By deleting repeated write data, the load bottleneck when data is written into the solid state disk is eliminated, and the writing performance of the solid state disk is improved. and space utilization. Background technique [0002] With the rapid development of the information technology revolution, big data and cloud computing have become the mainstream of today's era. The explosive growth of data and the continuous improvement of computer performance have put forward higher and higher requirements for storage systems. Storage systems are facing capacity and performance challenge. [0003] Data deduplication technology, which can be used to eliminate duplicate data in storage systems, is gradually considered to be an effective technology for reducing storage overhead and r...

AI Technical Summary

Problems solved by technology

However, since the pre-hash technology only calculates the fingerprint values ​​of data blocks that may be duplicated, a lot of deduplication rates are sacrificed

For example, in Figure Description 2, assuming that three identical data blocks M are continuously written, because when M is written for the first time, only the weak fingerprint value of M is calculated, and the fingerprint value of M is not calculated, so the second The second incoming M is deleted, which reduces the deduplication rate

[0017] To sum up, traditional deduplication relies on a very time-consuming fingerprint calculation process to find duplicate data. Although it can improve the service life of SSDs, the deduplication system will become the performance bottleneck of SSDs.

Therefore, traditional deduplication technology cannot be combined with SSD

In addition, although the pre-hash technology can reduce the overhead of fingerprint calculation, increase the throughput of deduplication, and improve the performance of SSDs, it reduces the deduplication rate.

Images