Ciphertext Deduplication via Convergent Encryption and HMAC
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data deduplication mechanisms cannot effectively deduplicate ciphertext while maintaining security, as encryption algorithms render identical plaintexts as distinct ciphertexts, making integrity checking and deduplication infeasible.
Innovation Solution
A system that uses a data storage engine and retrieval engine to determine unique ciphertext values by concatenating encrypted chunk hashes with message authentication codes, allowing for deduplication of ciphertext and integrity checking in a client-server data storage system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If encryption algorithms are used to protect data security, then data confidentiality is improved, but deduplication capability deteriorates because identical plaintexts are rendered as distinct ciphertexts
Solution Approach 1:
The patent segments data into fixed-size chunks before encryption, then applies convergent encryption where each chunk is encrypted independently using a key derived from the chunk's hash. This segmentation allows identical chunks to produce identical ciphertexts, enabling deduplication while maintaining security for non-identical data
Solution Approach 2:
The patent changes the encryption approach by using convergent encryption where the encryption key is derived from the data chunk itself (via hashing) rather than using unique keys for each encryption operation. This parameter change in the encryption scheme allows identical plaintext chunks to produce identical ciphertexts, resolving the contradiction between security and deduplication
2Quantity of substance
If convergent encryption is used to enable deduplication of ciphertext, then storage space efficiency is improved, but integrity checking capability deteriorates
Solution Approach 1:
The patent introduces HMAC (Hash-based Message Authentication Code) as an intermediary mechanism. The HMAC is computed over the ciphertext using a separate key, creating an authentication tag that verifies integrity without affecting the deduplication capability of the ciphertext itself. This intermediary layer restores integrity checking while preserving storage efficiency
3Reliability
If traditional encryption is used to maintain data security, then confidentiality is improved, but operational complexity increases due to inability to perform deduplication and integrity checking
Solution Approach 1:
The patent creates a multi-functional system where the encryption mechanism simultaneously provides confidentiality, enables deduplication through convergent encryption, and supports integrity checking via HMAC. This universal approach eliminates the need for separate mechanisms for each function, reducing operational complexity while maintaining security
Data Source
AI summary
Storing and retrieving ciphertext in data storage can include determining a first ciphertext value for a first data chunk to be saved to a client-server data storage system using an encrypted chunk hash value associated with the first data chunk as an initial value, and storing the first data chunk on a server in the client-server data storage system in response to determining that the first ciphertext value is a unique ciphertext value. Also, storing and retrieving ciphertext in data storage can include decrypting a ciphertext value for a second data chunk received from a client in the client-server data storage system and based on an encrypted chunk hash value associated with the second data chunk, and sending the second data chunk to the client in response to determining that the decrypted ciphertext value corresponds to an original data chunk saved to the server by the client.