Encrypted Data Deduplication via Distributed Ledger Hashing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional cryptographic data storage systems face inefficiencies in deduplication when dealing with encrypted data, particularly between hosts that do not trust each other, as encrypted data is treated as new data, leading to inefficient storage.
Innovation Solution
A method utilizing a distributed ledger, accessible to multiple hosts, to prevent duplication of encrypted data by comparing hash values of new data blocks with existing blocks, and if identical, transferring ownership of the existing block to the requesting host, thereby avoiding storage of duplicates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data deduplication techniques are used to reduce storage space by eliminating duplicate data, then storage efficiency is improved, but the technique becomes ineffective when data is encrypted because encrypted data is treated as new data even if it is a duplicate
Solution Approach 1:
The system performs preliminary hashing of data blocks before encryption. By computing hash values of plaintext data blocks before they are encrypted and stored, the system creates unique identifiers that remain consistent across duplicates. This preliminary action enables the deduplication system to recognize identical data even after encryption, as the hash values match before the encryption transformation occurs.
Solution Approach 2:
The patent introduces hash values as an intermediary mechanism between the data and the deduplication process. Instead of comparing encrypted data directly (which would fail due to different encryption keys), the system uses hash values of the original plaintext as a mediator to identify duplicates. The hash acts as a trusted intermediary that preserves identity information through the encryption process.
2Reliability
If conventional backup systems store entire changed data to ensure data recovery capability, then data security is improved, but storage space is wasted due to lack of deduplication
Solution Approach 1:
The system performs preliminary hashing of data blocks before encryption. By computing hash values of plaintext data blocks before they are encrypted and stored, the system creates unique identifiers that remain consistent across duplicates. This preliminary action enables the deduplication system to recognize identical data even after encryption, as the hash values match before the encryption transformation occurs.
Solution Approach 2:
The patent introduces hash values as an intermediary mechanism between the data and the deduplication process. Instead of comparing encrypted data directly (which would fail due to different encryption keys), the system uses hash values of the original plaintext as a mediator to identify duplicates. The hash acts as a trusted intermediary that preserves identity information through the encryption process.
Data Source
AI summary
Method prevents duplicate data to be stored in storage having hosts which do not trust each other. The method includes storage including encrypted data blocks, network further including distributed ledger, accessible to each of two or more hosts. The method includes steps, performed by first host when the first host wishes to store a new data block, obtaining first hash value for new data block, determining if first hash value is found in ledger. If first hash value is not found in ledger, creating set of confirmation data, encrypting new data block with block-unique encryption key, and storing first ownership record in ledger. If first hash value is found in ledger, obtaining information about second host that has already stored previous block identical to new data block, entering in ledger key request, receiving from ledger key transfer record, sending to ledger new ownership record.