Storage Array Chunk Encryption for Deduplication and Crypto-Erasure
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in ensuring data security and efficient storage space management due to the inability to perform data deduplication operations on encrypted data, leading to forensic recoverability of deleted data and inefficient use of storage resources.
Innovation Solution
Implementing a system that generates per-chunk keys and deduplication keys based on hash values, encrypts data chunks, and uses a key management server to manage per-logical-volume keys, enabling secure data deduplication and crypto-erasure without storing encryption keys on stable storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is encrypted before storage, then data security is improved, but data deduplication operations cannot be performed and storage efficiency deteriorates
Solution Approach 1:
The patent segments data into fixed-size chunks and generates unique keys for each chunk. This segmentation allows the system to process and encrypt data in manageable units while maintaining the ability to perform deduplication operations on the segmented data through hash value comparison, thus resolving the contradiction between security and efficiency.
Solution Approach 2:
The patent introduces hash values as an intermediary mechanism that enables deduplication operations without requiring decryption. By comparing hash values of encrypted data chunks, the system can identify duplicates and perform deduplication while data remains encrypted, thus maintaining both security and storage efficiency.
2Speed
If encryption keys are stored on stable storage for easy access, then data access speed is improved, but forensic recovery of deleted data becomes possible
Solution Approach 1:
The patent extracts encryption keys from stable storage and stores them only in volatile memory. This extraction eliminates the risk of forensic recovery from stable storage while maintaining fast access through volatile memory's speed, thus resolving the contradiction between access speed and forensic security.
Solution Approach 2:
The patent employs volatile memory as a disposable, short-living storage medium for encryption keys. Volatile memory automatically loses its contents when power is removed, providing forensic security while maintaining the speed characteristics needed for efficient data access during operational periods.
3Productivity
If data deduplication is performed on unencrypted data, then storage space optimization is improved, but data security deteriorates
Solution Approach 1:
The patent performs deduplication operations on encrypted data chunks before final storage by comparing hash values. This preliminary action allows storage space optimization to occur while data remains encrypted, thus achieving both storage efficiency and security simultaneously rather than requiring decryption first.
Data Source
AI summary
A method for operating a storage array includes: generating a per-chunk key and a deduplication key for a data chunk based on a hash value, wherein the per-chunk key is stored in a metadata file; obtaining a per-volume key for a volume of the storage array; encrypting a metadata folder associated with the volume to generate an encrypted metadata folder using the per-volume key, in which the encrypted metadata folder comprises at least the metadata file; compressing the data chunk to generate a compressed data chunk (CDC); encrypting the CDC to generate an encrypted compressed data chunk (ECDC) using the per-chunk key, in which the ECDC is stored in the volume; obtaining storage location information of the ECDC; updating a hash value lookup table based on the storage location information; and initiating notification of a user of a client about completion of a write request.