Encrypted Data Deduplication via Deterministic Key Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional encryption techniques in information handling systems prevent deduplication of encrypted data due to the use of different initialization vectors and keys, making it impossible for server devices to efficiently store and manage duplicate data without access to unencrypted data or encryption keys.
Innovation Solution
The system performs a hash operation on data to generate a Data Encryption Key (DEK), encrypts the data using the DEK, and then uses a Key Encryption Key (KEK) to encrypt the DEK, allowing for deduplication of encrypted data without the server needing access to the unencrypted data or keys, while enabling cryptoshredding for secure data deletion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional encryption techniques are used with different initialization vectors for each encryption operation, then data security is improved, but deduplication capability deteriorates
Solution Approach 1:
The encryption process is segmented into two distinct stages: first, deterministic encryption using a data-derived key (DEK) that produces identical ciphertext for identical plaintext, enabling deduplication; second, randomization using a randomization key (RK) that adds randomness to protect security. This segmentation allows each stage to fulfill its specific function without compromising the other.
Solution Approach 2:
The patent implements nested encryption layers where the DEK encryption is nested within an additional randomization layer. The deterministic encrypted data is further processed with a randomization key, creating a nested structure that preserves deduplication capability at the inner layer while adding security at the outer layer.
2Reliability
If server devices store encrypted data without access to encryption keys, then data security is improved, but the ability to perform deduplication operations deteriorates
Solution Approach 1:
The encrypted data structure is designed to be self-sufficient for deduplication purposes. The deterministic encryption layer produces consistent ciphertext that can be compared and deduplicated without requiring key access, allowing the storage system to perform deduplication operations autonomously on the encrypted data itself.
Solution Approach 2:
The patent changes the encryption parameters by using a deterministic encryption mode that produces consistent output for identical input, rather than traditional randomized encryption. This parameter change enables the encrypted data to maintain properties that facilitate deduplication while still being secure.
3Reliability
If multiple client devices encrypt the same data with different encryption keys, then data security is improved, but storage efficiency deteriorates
Solution Approach 1:
The deterministic encryption function serves multiple purposes: it provides security through encryption while simultaneously enabling deduplication across multiple client devices. Different clients encrypting the same data with their respective keys produce different deterministic ciphertexts, but the structure allows the server to identify and deduplicate based on the underlying data hash, achieving both security and storage efficiency.
Solution Approach 2:
The patent introduces a data hash as an intermediary that mediates between the encrypted data from multiple clients and the deduplication process. The hash of the original plaintext serves as a common reference point that allows the server to identify duplicate data across different clients without needing to decrypt or access their individual encryption keys.
Data Source
AI summary
An encrypted data storage system includes a storage system that is configured to store encrypted data, and a first client device that is coupled to the storage system. The first client device performs a hash operation on first data to generate a Data Encryption Key (DEK), and uses the DEK to perform a data encryption operation on the first data to generate encrypted first data. The first client device then uses a first Key Encryption Key (KEK) to perform a first key encryption operation on the DEK to generate a first encrypted DEK, associates the first encrypted DEK with the encrypted first data, and transmits the encrypted first data to the storage system for storage.