Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Weak fingerprint repeated data deletion mechanism based on flash memory solid-state disk

A technology of data deduplication and solid-state disk, applied in the direction of electrical digital data processing, input/output process of data processing, instruments, etc., can solve the performance bottleneck of deduplication system solid-state disk, uncalculated M fingerprint value, sacrifice deduplication rate and other issues, to achieve the effects of improving life and space utilization, fewer weak fingerprint digits, and lower fingerprint calculation overhead

Inactive Publication Date: 2016-09-07
NAT UNIV OF DEFENSE TECH
View PDF3 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, since the pre-hash technology only calculates the fingerprint values ​​of data blocks that may be duplicated, a lot of deduplication rates are sacrificed
For example, in Figure Description 2, assuming that three identical data blocks M are continuously written, because when M is written for the first time, only the weak fingerprint value of M is calculated, and the fingerprint value of M is not calculated, so the second The second incoming M is deleted, which reduces the deduplication rate
[0017] To sum up, traditional deduplication relies on a very time-consuming fingerprint calculation process to find duplicate data. Although it can improve the service life of SSDs, the deduplication system will become the performance bottleneck of SSDs.
Therefore, traditional deduplication technology cannot be combined with SSD
In addition, although the pre-hash technology can reduce the overhead of fingerprint calculation, increase the throughput of deduplication, and improve the performance of SSDs, it reduces the deduplication rate.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Weak fingerprint repeated data deletion mechanism based on flash memory solid-state disk
  • Weak fingerprint repeated data deletion mechanism based on flash memory solid-state disk
  • Weak fingerprint repeated data deletion mechanism based on flash memory solid-state disk

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0032] figure 1 It is a schematic diagram of the principle of data deduplication, including data block, calculation of fingerprint value, query index table, deletion of duplicate blocks and storage of unique blocks.

[0033] figure 2 It is a schematic diagram of the principle of the pre-hash deduplication mechanism, including the calculation of weak fingerprints and the calculation of SHA-1 fingerprints. In the figure, three identical data blocks M are continuously written, and the second M cannot be deleted.

[0034] image 3It is a structural diagram of the weak fingerprint deduplication mechanism based on the flash memory solid state disk used in the present invention, and the weak fingerprint deduplication mechanism is implemented in the flash translation layer (Flash Translation Layer) inside the solid state disk.

[0035] Figure 4 It is a schematic diagram of the principle of the weak fingerprint deduplication mechanism. The specific execution process is:

[0036]...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention relates to a weak fingerprint repeated data deletion mechanism based on a flash memory solid-state disk. Oriented to online applications with repeated data, the mechanism implements a deduplication system in a solid-state disk by using hardware resources, and deduplication is performed through weak fingerprint calculation and byte direct comparison, so that the defects of the conventional deduplication technology and pre-hash deduplication technology are overcome, a higher deduplication throughput and deduplication rate are achieved, and overheads on the aspects of time, space, hardware resources and the like are minimized. Moreover, due to a low hardware overhead, parallelism in the solid-state disk can be fully used, and a plurality of deduplication parts perform deduplication in parallel, so that a write delay is greatly reduced. The mechanism is capable of combining with a flash memory solid-state disk properly, is capable of effectively eliminating repeated data, and improves the service life and performance of the solid-state disk.

Description

technical field [0001] The present invention is applicable to the technical field of data deduplication, and provides a weak fingerprint deduplication mechanism based on a flash memory solid state disk. By deleting repeated write data, the load bottleneck when data is written into the solid state disk is eliminated, and the writing performance of the solid state disk is improved. and space utilization. Background technique [0002] With the rapid development of the information technology revolution, big data and cloud computing have become the mainstream of today's era. The explosive growth of data and the continuous improvement of computer performance have put forward higher and higher requirements for storage systems. Storage systems are facing capacity and performance challenge. [0003] Data deduplication technology, which can be used to eliminate duplicate data in storage systems, is gradually considered to be an effective technology for reducing storage overhead and r...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06
CPCG06F3/0641G06F3/0608G06F3/061G06F3/0616
Inventor 肖侬陈正国陈志广刘芳陈微欧洋
Owner NAT UNIV OF DEFENSE TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products