Encrypted Segment Deduplication Using Pre-Encryption Hashing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing segment deduplication systems are inefficient when data is encrypted or compressed, as encryption and compression prevent the identification of identical segments, thereby eliminating the efficiency of deduplication.
Innovation Solution
A system that processes encrypted and compressed segments by breaking data streams or files into segments, determines if they have been previously stored, and stores or retrieves them while decrypting and decompressing as necessary to maintain efficient deduplication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is encrypted before storage, then security is improved, but the ability to identify identical segments is lost, eliminating deduplication efficiency
Solution Approach 1:
The system performs preliminary hashing on the plaintext data before encryption to create a fingerprint that can be used for deduplication identification. This preliminary action allows the system to identify identical segments even after encryption, resolving the contradiction between security and deduplication efficiency.
Solution Approach 2:
The patent introduces a hash value as an intermediary between the plaintext data and the encrypted data. The hash serves as a mediator that preserves the ability to identify identical segments without compromising security, as the hash is computed on the plaintext before encryption occurs.
2Quantity of substance
If compression is applied to data, then storage efficiency is improved, but the ability to identify identical segments is lost, eliminating deduplication efficiency
Solution Approach 1:
The system computes hash values on the plaintext data before compression is applied. This preliminary hashing action enables the system to identify identical segments based on their original content, even after compression transforms the data representation, thus maintaining deduplication efficiency while achieving storage efficiency through compression.
3Productivity
If deduplication is implemented without encryption, then storage efficiency is improved, but data security is compromised
Solution Approach 1:
The system performs deduplication identification through hashing before encryption is applied to the data. This sequence allows the system to achieve storage efficiency by identifying and eliminating duplicate segments while then encrypting the stored data to maintain security, thus resolving the contradiction between storage efficiency and data security.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system for storing encrypted data comprises a processor and a memory. The processor is configured to receive an encrypted segment. The encrypted segment is determined by breaking a data stream, a data block, or a data file into one or more segments and encrypting each of the one or more segments. The processor is further configured to determine whether the encrypted segment has been previously stored, and in the event that the encrypted segment has not been previously stored, store the encrypted segment. The memory is coupled to the processor and configured to provide the processor with instructions.