Storage Array Deduplication with Per-Chunk Encryption and Crypto-Erasure
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face challenges in ensuring data security and efficient storage space management due to the inability to perform data deduplication operations on encrypted data, leading to forensic recoverability of deleted data and inefficient use of storage resources.
Innovation Solution
Implementing a system that generates per-chunk keys and deduplication keys based on hash values, encrypts data chunks, and uses a key management server to manage per-logical-volume keys, enabling secure data deduplication and crypto-erasure without storing encryption keys on stable storage devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is encrypted for security, then data security is improved, but data deduplication operations cannot be performed
Solution Approach 1:
The patent segments data into fixed-size chunks and generates unique per-chunk keys for each segment. This segmentation allows deduplication operations to be performed on individual chunks independently, enabling efficient storage while maintaining encryption security. Each chunk can be encrypted, deduplicated, and managed separately without affecting other data segments.
Solution Approach 2:
The patent introduces per-chunk keys as intermediary elements between the plaintext data and the encryption process. These keys are derived from hash values of the data chunks and serve as mediators that enable both security (through encryption) and deduplication (through key comparison) operations without requiring direct access to the encrypted data.
2Ease of operation
If encryption keys are stored on stable storage devices for accessibility, then data accessibility is improved, but forensic recovery of deleted data becomes possible
Solution Approach 1:
The patent extracts the encryption keys (per-chunk keys) from stable storage devices and stores them only in volatile memory. This extraction eliminates the security vulnerability of having keys persist on disk, preventing forensic recovery of deleted encrypted data while maintaining accessibility during system operation.
Solution Approach 2:
The patent employs volatile memory as a disposable, short-lived storage medium for encryption keys. Unlike permanent storage devices, volatile memory automatically clears data when power is removed, providing inherent security against forensic recovery while maintaining ease of access during system operation.
3Productivity
If data is compressed and deduplicated to optimize storage space, then storage efficiency is improved, but data security is compromised
Solution Approach 1:
The patent performs deduplication operations using per-chunk keys derived from hash values before the actual encryption process. This preliminary deduplication action identifies and eliminates duplicate data segments based on their content hash, ensuring storage efficiency is achieved before security-sensitive encryption operations occur.
Solution Approach 2:
The patent applies different processing qualities to different parts of the data: deduplication operations use hash-based comparison for efficiency, while encryption operations use secure cryptographic methods. This local quality differentiation allows each operation to use the most appropriate method for its specific purpose without compromising overall security or efficiency.
Data Source
AI summary
A method for operating a storage array, the method includes analyzing a read request to identify an encrypted metadata folder associated with a volume of the storage array, obtaining a per-volume key for the volume, decrypting the encrypted metadata folder to obtain a metadata folder using the per-volume key, analyzing the metadata folder to identify a metadata file of a data chunk of data, extracting a per-chunk key associated with the data chunk from the metadata file, identifying storage location information of the data chunk based on a hash value lookup table, decrypting the data chunk using the per-chunk key to obtain a decrypted data chunk, decompressing the decrypted data chunk to obtain a decompressed data chunk (DDC), and initiating transmission of the DDC to the client.