An Efficient Metadata Management Method in Encrypted Data Deduplication System

Abstract

The invention discloses an efficient metadata management method in an encrypted repeated data deletion system, belonging to the technical field of data storage. The present invention can be used in a client-server storage model, and can also be applied to cloud computing. The present invention collects metadata of adjacent encrypted data blocks into metadata blocks, that is, each metadata block corresponds to an encrypted data block of an area, and only stores metadata of metadata blocks in file metadata and key metadata. Related Information. At the same time, the present invention further applies the deduplication operation to metadata blocks: since the same data blocks across backups are often aggregated to form duplicate areas, only one metadata block needs to be reserved for these duplicate data areas, thereby reducing metadata storage overhead.

Description

technical field [0001] The invention belongs to the technical field of data storage, and in particular relates to metadata management oriented to an encryption deduplication system. Background technique [0002] Data deduplication technology is widely used in backup storage systems, and the technical solutions usually used include: ① Divide the backup file into logical data blocks of fixed or variable length; ② For each logical data block, calculate the hash based on the data content value (called a fingerprint) to uniquely identify the corresponding data block; ③ only store a unique copy (called a physical data block) of a logical data block with the same fingerprint to save storage space. [0003] To protect data privacy, cryptographic deduplication adds an encryption layer that acts on logical data blocks. The encryption layer generates an encryption key based on the data content of the logical data block, called the MLE (message-locked encryption) key. For example, the ...

AI Technical Summary

Problems solved by technology

[0011] Currently, in the existing metadata management technology, the literature "E.Kruus, C.Ungureanu, and C.Dubnicki. Bimodal content defined chunking for backup streams.In Proc.ofUSENIX FAST,2010" and "D.R.Bobbarjung, S.Jagannathan, and C.Dubnicki.Improvingduplicate elimination in storage systems.ACM Transactions on Storage,vol.2,no.4,pp.424–448,2006"By constructing real-time repeated queries on fingerprint indexes, and several adjacent duplicates Chunks are merged into a superchunk to reduce file metadata overhead; however, these methods cannot compress key metadata because each chunk still needs to be encrypted with a key derived from its own content

Document "J.Li, X.Chen, M.Li, J.Li, P.P.C.Lee, and W.Lou. Secure deduplication with efficient and reliable convergent key management. IEEE Transactions on Parallel Distributed Systems, vol.25, no.6, pp .1615–1625, 2014" applies deduplication directly to the key to reduce key metadata overhead; however, since the length of the key is usually comparable to the length of the fingerprint index entry (both tens of bytes), this The additional metadata overhead caused by key-based deduplication offsets the storage savings for key metadata

Document "D. Meister, Andre Brinkmann, and T. Su . β.File recipe compression in datadeduplication systems.In Proc.of USENIX FAST,2013"proposed four file metadata compression methods to replace file metadata entries with short codewords; however, these methods either cannot be applied by the file owner The key metadata encrypted by the master key, or the metadata that can only reduce zero blocks

Images