Encrypted Backup Deduplication Across Untrusted Hosts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data deduplication techniques are inefficient for encrypted data storage systems where there is no mutual trust between hosts, leading to unnecessary storage occupation and increased costs due to duplicate data storage.
Innovation Solution
A method utilizing a distributed ledger, such as a blockchain, to manage and verify ownership of encrypted data blocks, preventing duplication by comparing hash values and generating backup set description records to identify and encrypt data blocks, ensuring deduplication even among untrusted hosts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional data deduplication techniques are used for encrypted data storage, then data reduction can be achieved under mutual trust conditions, but deduplication fails and storage space is wasted when there is no mutual trust between hosts
Solution Approach 1:
The patent introduces a trusted third party (TTP) as an intermediary that issues certificates to hosts. This certificate mechanism allows hosts without mutual trust to prove their identity and ownership of encrypted data, enabling the storage system to perform deduplication by comparing certificates and data hashes rather than relying on host-to-host trust relationships.
Solution Approach 2:
The patent replaces the mechanical trust-based verification system with a cryptographic certificate-based system. Instead of hosts trusting each other directly, the system uses digitally signed certificates and hash comparisons to verify data uniqueness, substituting interpersonal trust with mathematical proof mechanisms.
2Reliability
If encrypted data is stored without deduplication, then security is maintained, but storage costs and performance burden increase due to duplicate data
Solution Approach 1:
The patent extracts only the essential verification elements (certificates and data hashes) from the full encrypted data blocks. By storing and comparing these extracted features rather than the complete encrypted data, the system enables deduplication without requiring decryption, thus maintaining security while reducing storage costs.
Solution Approach 2:
The patent changes the verification parameter from full encrypted data comparison to hash value comparison. This parameter transformation allows the system to identify duplicate encrypted data blocks efficiently without compromising security, as hash functions provide unique fingerprints for encrypted data while being computationally efficient to compare.
3Reliability
If conventional backup systems store entire changed data, then data recovery is ensured, but storage performance deteriorates and costs increase over time
Solution Approach 1:
The patent segments the backup process into two independent stages: verification stage (comparing certificates and hashes to identify duplicates) and storage stage (storing only unique data blocks). This segmentation allows the system to quickly identify and eliminate duplicates before actual data transfer and storage, improving backup performance while ensuring complete data recovery capability for all unique blocks.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A method of preventing duplication of encrypted data in storage arranged in network in which two or more hosts can store data. The storage includes one or more encrypted data blocks, and the network further includes a distributed ledger accessible to each of the two or more hosts. The method leverages the distributed ledger for efficient deduplication of the data even with untrusted parties, such as the hosts. The method, performed by first host, includes obtaining a first hash value, a proof of ownership, and a random key for each block in plurality of new blocks of data. The method further includes generating and storing backup set description record, storing indication of the backup set description record, and determining zero or more blocks in plurality of new blocks that are present in storage to prevent duplication.