Safety data repetition removing method and system applicable to backup system

Abstract

The invention discloses a safety data repetition removing method applicable to a backup system. The method includes the following steps that a backup request submitted by a user is received; all files needing backup are partitioned and multiple data blocks of different sizes are obtained; the Hash algorithm is used for calculating the Hash value F1 of each data block, wherein the Hash values F1 serve as encrypting keys of the data blocks; the Hash algorithm is used again for calculating the Hash value F2 of the Hash value F1 of each data block, wherein the Hash values F2 serve as fingerprints of the data blocks to identify repeated data blocks and the classic encryption algorithm and a private key of the user are used for encrypting the Hash values F1 of the data blocks to obtain ciphertexts E (F1) of the Hash values F1 of the data in order to protect the encrypting keys of the data blocks. All the data block fingerprints F2 and the like are packaged in sequence into fingerprint segments which are transmitted to a storage server. The SSL protocol is adopted in all communication processes. According to the safety data repetition removing method applicable to the backup system, the safety data repetition removing method is adopted to ensure that storage safety problem that data are lost or tampered is solved under the condition that the data repetition removing rate is not changed.

Description

technical field [0001] The invention belongs to the field of computer storage, and more specifically relates to a method and system for deduplication of secure data suitable for backup systems. Background technique [0002] In recent years, with the development and popularization of computer technology and networks, the amount of data information storage in the world has shown an explosive growth trend. The popularity of personal computers, smartphones and other devices around the world, and the growing Internet access in emerging markets, as well as the explosion of data generated by devices such as surveillance cameras or smart meters, have doubled the size of the digital universe in the past two years. Fan, reaching an astonishing 2.8ZB. IDC predicts that by 2020, the scale of the digital universe will exceed expectations, reaching 40ZB. The growth of online digital content has led to an increase in the cost of storage systems including hardware and human maintenance. I...

AI Technical Summary

Problems solved by technology

If according to the classic data deduplication algorithm, different users encrypt the shared data blocks, due to the randomness of the ciphertext, they will get different ciphertexts and generate different fingerprints, which will greatly reduce the efficiency of data deduplication

If different users calculate and process the plaintext of shared data blocks to obtain the same fingerprint, and then use different keys to encrypt data blocks to obtain different ciphertexts, other users cannot decrypt them.

At the same time, adding security technologies such as encryption and identity authentication to data deduplication technology will inevitably increase the burden on the backup system and reduce the performance of the backup system

Images