Method and system for deleting global repeating data and storage device

A deduplication, global technology, applied in the input/output process of data processing, electrical digital data processing, special data processing applications, etc., can solve the problems of reducing system access performance, reducing system read and write access performance, etc. Access performance, reducing random write operations, and improving read performance

Active Publication Date: 2014-01-15
易乐天 +2
View PDF4 Cites 45 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This method still has the following problems: 1. This method requires the transfer of all data to be written between the data redirection module and the controller, so that the controller can complete the detection of duplicate data and the redundant operation
Since frequent large-scale data transmission will occupy a large amount of controller bandwidth, th...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for deleting global repeating data and storage device
  • Method and system for deleting global repeating data and storage device
  • Method and system for deleting global repeating data and storage device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0041] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0042] Such as figure 1 As shown, the method for global deduplication of the present invention comprises the following steps:

[0043] 1.1 Divide the management layer into several management nodes, divide the range of acceptable fingerprint values ​​for each management node, and establish a unique The mapping relationship; each management node establishes a fingerprint value index structure for the fingerprint value that has a mapping relationship with it, and writes the fingerprint value index structure into the storage device or storage medium;

[0044] 1.2 The receiving layer segments the received data stream to obtain multiple written data segments, calculates the written data segment fingerprint value for each written data segment, and searches for the management node corresponding to the written data segment fingerprint value ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a method and system for deleting global repeating data and a storage device. The method comprises the following steps that 1.1, initialization is conducted; 1.2, a data stream is divided into a plurality of read-in data segments through a receiving layer, a read-in data segment fingerprint value is calculated for each read-in data segment, a management node corresponding to each read-in data segment fingerprint value is looked up, and the read-in data segment fingerprint values are sent to the management nodes; 1.3, whether the received read-in data segment fingerprint values exist in a fingerprint value index structure or not is judged through the management nodes, if yes, the read-in data segment is directly written in the storage device or a storage medium through the receiving layer, and the fingerprint value index structure is updated; if not, updating is conducted directly. The system is used for achieving the method. The storage device comprises the storage medium and a storage controller. The storage controller comprises the system for deleting the global repeating data. According to the method and system for deleting the global repeating data and the storage device, only the fingerprint values of the data segments need to be transmitted, not all the data segments need to be transmitted, and the interactive operation performance is greatly improved through establishment of the fingerprint value index structure and partition of a fingerprint value management range.

Description

technical field [0001] The present invention mainly relates to the field of data storage, in particular to a method and system for global deduplication of data storage devices. Background technique [0002] With the explosive growth of data volume, the amount of data stored in the storage system is increasing. According to the statistics of IDC, the total amount of global data reached trillions of GB in 2012, and more than 95% of the data is unstructured data; In many data-centric computing centers, the amount of new data generated every day has reached 100GB or even 1TB. At the same time, new storage media and their technologies, such as flash memory and phase change memory, are also developing. As a typical new type of storage medium, flash memory storage medium has the characteristics of high density, light weight, and low energy consumption, and is an ideal storage medium to replace the disk in the main storage system. The minimum read / write unit of flash memory is a f...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30G06F3/06
CPCG06F3/0641G06F16/215G06F16/2255
Inventor 易乐天钱凯赵朕毅
Owner 易乐天
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap