Single Instance Storage for Encrypted Data Using Pre-Encryption Hashing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Single instance storage (SIS) techniques do not work well with encrypted data, as the encryption process randomizes data bits, making it unlikely for encrypted data blocks to be identical, thus preventing the service provider from performing single instancing without decrypting the data, which is undesirable and often not allowed by clients.
Innovation Solution
Using a hash value computed from unencrypted data to reference a single instance of encrypted data, allowing the client to store and retrieve encrypted data blocks while maintaining context information for decryption, without revealing the decryption key to the service provider, enabling duplicate detection and storage in an encrypted form.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is encrypted before storage, then data security is improved, but duplicate detection capability deteriorates because encryption randomizes data bits
Solution Approach 1:
The system computes and stores hash values of the original unencrypted data blocks before encryption occurs. This preliminary action creates a reference fingerprint that remains consistent regardless of subsequent encryption operations, enabling duplicate detection without requiring decryption of stored data.
Solution Approach 2:
The patent introduces hash values as an intermediary element between the original data and its encrypted form. This intermediary serves as a mediator that allows the storage system to identify duplicates without accessing the actual encrypted data content, thus maintaining both security and duplicate detection capability.
2Reliability
If the service provider stores encrypted data, then data confidentiality is improved, but the ability to perform single instance storage deteriorates
Solution Approach 1:
The system extracts the identification function from the encrypted data itself and places it in the hash value that is stored separately. By taking out the duplicate-detection capability from the encrypted data blob, the system enables single instance storage operations without requiring the service provider to decrypt or access the confidential encrypted data.
Solution Approach 2:
Instead of storing and comparing encrypted data blocks directly, the system stores and compares hash value copies of the original data. These hash copies serve as surrogates that enable single instance storage operations while the actual encrypted data remains confidential and untouched by comparison operations.
3Ease of operation
If decryption keys are stored with encrypted data, then data retrieval capability is improved, but security against key breaches deteriorates
Solution Approach 1:
The system segments the data retrieval process into two independent parts: hash value comparison for duplicate detection and key management for decryption. This segmentation allows the service provider to perform retrieval operations using only hash values, while key management remains separate and can be handled through secure key exchange protocols without exposing keys to the storage system.
Data Source
AI summary
Described is a technology by which data is efficiently and securely stored in a single instance store. A hash value is computed from data in its unencrypted form. The hash value is used to reference a single instance of that data when stored in its encrypted form. In this manner, duplicate data blocks are detectable independent of their encryption, yet stored in an encrypted form in a single instance store. In one aspect, context information for decrypting the encrypted data is stored in association with the data. When the client wants to restore the data, the client sends the hash value for that block to the single instance store service. The service returns the block and the associated context information. The client uses the context information to decrypt the block. For example, the context may comprise a key identifier which the client uses to lookup the correct key.