Secure Data Deduplication via Fixed-Size Block Chunking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Variable-sized block-based deduplication techniques leak significant information about file contents, compromising data confidentiality and security, and are vulnerable to attacks where adversaries can guess files based on block sizes.
Innovation Solution
The method employs a cryptographic hash to check for existing files, uses AES-based chunking with a secret key that is protected by trusted computing or hardware dongles, and involves a keyserver to generate encryption keys, ensuring secure deduplication without revealing content information, and supports secure deletion and assisted deletion mechanisms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If variable-sized block-based deduplication is used to maximize storage savings, then storage efficiency is improved, but information about file contents is leaked through block size patterns
Solution Approach 1:
The patent changes the parameter of block size from variable to fixed, eliminating the information leakage about file contents while maintaining deduplication functionality. By using fixed-size blocks instead of variable-size blocks that adapt to content patterns, the system prevents adversaries from inferring information about stored files through block size analysis.
Solution Approach 2:
The patent introduces fixed-size placeholder blocks that serve as temporary or disposable structures during the deduplication process. These fixed-size blocks can be uniformly stored and managed without revealing content information, effectively masking the true variable nature of deduplicated data while maintaining storage efficiency.
2Loss of information
If fixed-sized blocks are used for deduplication to protect confidentiality, then data confidentiality is improved, but storage savings are reduced
Solution Approach 1:
The patent introduces fixed-size placeholder blocks as an intermediary structure between the variable content and the storage system. These placeholders maintain fixed sizes for confidentiality protection while the actual deduplication logic operates on the underlying variable content, allowing both confidentiality and storage efficiency to coexist.
Solution Approach 2:
The patent segments the storage system into multiple layers: fixed-size placeholder blocks for external storage and transmission, and variable-size actual data blocks for internal deduplication. This segmentation allows each layer to optimize for its specific purpose while maintaining overall system efficiency and security.
3Loss of information
If file-based deduplication is used to maintain security, then data confidentiality is preserved, but storage savings are limited compared to block-based methods
Solution Approach 1:
The patent segments files into fixed-size blocks that can be independently processed and stored. This segmentation enables block-level deduplication operations while maintaining fixed-size boundaries that protect confidentiality, combining the benefits of both file-based and block-based approaches.
Solution Approach 2:
The patent introduces a new dimension of fixed-size placeholders that sits between the content and the storage system. This additional layer operates in a different dimensional space, allowing variable content to be represented through fixed-size structures, thereby enabling both fine-grained deduplication and confidentiality protection.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention relates to a method for storing a data file, 'DF' on a storage entity, 'SE', comprising the steps of: a) Chunking said DF into a number of blocks using a one-way- function and a chunking key by a computing entity, 'CE', b) Computing a hash value for each of said blocks by said CE, c) Checking, by one or more proxies, 'PE', if said blocks are already stored resulting in a first number of already stored blocks and a second number of blocks not being stored, d) Encrypting, by said CE, the blocks not being stored using an encryption key, e) Transmitting, by said CE, said encrypted blocks to said SE for storing and f) Informing, by said CE, the PE about the hash values of the transmitted and corresponding storage location information of said transmitted blocks.