Secure Data Deduplication via Key Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data deduplication techniques face challenges when applied to encrypted data, as different encryption keys for different users and files result in non-matching hash values, preventing effective deduplication and compromising data security.
Innovation Solution
Generate a key from plaintext data, encrypt it, and compare it to stored keys; if a match is found, the encrypted data is discarded, and a reference is stored, allowing for deduplication while maintaining data security by associating the key with the matching encrypted data element.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If different encryption keys are used for different users and files, then data security is improved, but data deduplication capability deteriorates because hash values do not match
Solution Approach 1:
The patent extracts the encryption key from the encrypted data block and uses it as the basis for deduplication identification. Instead of hashing the encrypted data directly (which would be key-dependent), the system separates the key extraction step and uses the key itself as the identifier for duplicate detection, allowing deduplication to work across different encryption keys.
Solution Approach 2:
The patent introduces an intermediary mechanism where the encryption key serves as a mediator between the encrypted data and the deduplication process. The key is extracted and used as an intermediate identifier to match duplicate data blocks, enabling the system to identify duplicates even when different encryption keys are applied to the same underlying plaintext.
2Reliability
If data encryption is applied before storage, then data security is improved, but storage optimization through deduplication deteriorates because encrypted data cannot be compared for duplicates
Solution Approach 1:
The patent extracts the encryption key from each encrypted data block and uses this extracted key as the basis for deduplication. By separating the key extraction from the encrypted data storage, the system can identify duplicate keys and avoid storing redundant encrypted blocks, thereby achieving storage optimization while maintaining encryption security.
Solution Approach 2:
Instead of copying and storing duplicate encrypted data blocks, the patent creates references or pointers to the original stored block when a duplicate key is detected. This allows multiple users or files to reference the same encrypted data storage location, significantly reducing storage space consumption while maintaining the ability to retrieve data through the reference mechanism.
3Productivity
If hash values are compared for deduplication, then duplicate detection is improved, but data security deteriorates because hash comparison requires plaintext or unencrypted data access
Solution Approach 1:
The patent inverts the traditional deduplication approach by not hashing the data content itself. Instead, it uses the encryption key (which is derived from or associated with the data) as the identifier for duplicate detection. This inversion allows the system to perform deduplication operations on encrypted data without ever exposing the plaintext, thereby maintaining both duplicate detection accuracy and data security.
Solution Approach 2:
The encryption key acts as an intermediary that enables duplicate detection without requiring access to the actual data content. By using the key as a mediator for identification and comparison, the system can determine duplicates while keeping the underlying data encrypted and secure, eliminating the need to decrypt data for comparison purposes.
Data Source
AI summary
Approaches for deduplicating data include generating a first key from plain text data of a first data element. The first data element is encrypted using the first key. The first key is compared to each key of a plurality of previously stored keys, which are associated with other encrypted data elements. In response to the first key matching a second key of the plurality of previously stored keys, the encrypted first data element is compared to the other encrypted data element associated with the second key. In response to the encrypted first data element matching the other encrypted data element, the first key is associated with the other encrypted data element, and the encrypted first data element is discarded.