Storage method for redundancy deletion block device based on location-sensitive hash

Abstract

The invention discloses a storage method for a redundancy deletion block device based on location-sensitive hash, and belongs to the data storage field. The method comprises the following steps: putting data blocks of redundant writing detection operation and a corresponding digital finger print into the current operating queue; D: judging whether the number of the data blocks in the queue exceeds threshold value or not, if so, taking threshold value of data blocks as a data section, and executing the step F, and otherwise, executing the step E; E: judging whether the data block at the front of the queue is overtime or not, if so, taking the data blocks as the data section, and executing the step F, and otherwise, executing the step D; F: judging whether the set of metadata of similar data sections exists or not, if so, executing the step G, and otherwise, establishing an empty set, and executing the step G; and G: orderly judging whether digital finger prints of data blocks exist in the set of metadata of the similar data sections or not, if so, modifying the memory addresses of the data blocks, and otherwise, generating the metadata of the data blocks. The method reduces the time of accessing the metadata in the redundant writing detection operation process.

Description

technical field

[0001] The invention relates to the technical field of data storage, in particular to a storage method for a redundant block device based on position-sensitive hashing. Background technique

[0002] With the explosive growth of digital information, the space occupied by data is getting bigger and bigger; in the past 10 years, the capacity of storage systems provided by many industries has grown from tens of GB to hundreds of TB, or even several PB, which has fully doubled. More than 10,000 times. With the exponential growth of data, enterprises are faced with more and more time points for quick backup and recovery, the cost of managing and saving data is getting higher and higher, and the space and power consumption of data centers are also increasing. The study found that up to 60% of the data stored in the application system is redundant, and it becomes more and more serious as time goes by.

[0003] In order to alleviate the problem of space growth in th...

Examples

Embodiment Construction

[0035] The specific implementation manners of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0036] figure 1 It is a flow chart of the location-sensitive hash-based redundant block device storage method of the present invention. Such as figure 1 As shown, the method comprises the steps of:

[0037] A: Add a redundant write operation at the block device layer to determine whether the type of the current write operation is a redundant write operation. If so, perform step B; otherwise, directly write the data of the current write operation into the secondary storage, that is Perform an existing write operation. In the step A, the original read and write operations are not changed, and it can be guaranteed that the existing storage system that does not support red...

technical field

[0001] The invention relates to the technical field of data storage, in particular to a storage method for a redundant block device based on position-sensitive hashing. Background technique

[0002] With the explosive growth of digital information, the space occupied by data is getting bigger and bigger; in the past 10 years, the capacity of storage systems provided by many industries has grown from tens of GB to hundreds of TB, or even several PB, which has fully doubled. More than 10,000 times. With the exponential growth of data, enterprises are faced with more and more time points for quick backup and recovery, the cost of managing and saving data is getting higher and higher, and the space and power consumption of data centers are also increasing. The study found that up to 60% of the data stored in the application system is redundant, and it becomes more and more serious as time goes by.

[0003] In order to alleviate the problem of space growth in th...

Examples

Embodiment Construction

[0035] The specific implementation manners of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0036] figure 1 It is a flow chart of the location-sensitive hash-based redundant block device storage method of the present invention. Such as figure 1 As shown, the method comprises the steps of:

[0037] A: Add a redundant write operation at the block device layer to determine whether the type of the current write operation is a redundant write operation. If so, perform step B; otherwise, directly write the data of the current write operation into the secondary storage, that is Perform an existing write operation. In the step A, the original read and write operations are not changed, and it can be guaranteed that the existing storage system that does not support red...

Images