Cloud Storage Gateway Deduplication via User-Agnostic Encryption
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional cloud storage systems face challenges in minimizing data duplication and bandwidth utilization while ensuring data security, as they either store duplicate encrypted files or compromise security by managing encryption keys on the server side.
Innovation Solution
A cloud storage system that employs user-agnostic file encryption and deduplication, where files are encrypted with a unique key that produces identical encrypted data across users, and a cloud storage gateway manages file chunks to reduce bandwidth usage by checking for existing data before uploading, using a file system manifest to track and manage file chunks efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the service encrypts data on the server side to enable deduplication, then storage efficiency is improved, but data security deteriorates as the operator gains access to user data
Solution Approach 1:
A key derivation function acts as an intermediary between the encryption key and the encrypted data. The same encryption key produces different encrypted outputs through the key derivation function, allowing the service to store only one copy of the transformed data while users cannot reverse the transformation to access their data. This mediator enables both deduplication and security simultaneously.
Solution Approach 2:
The patent transforms the encryption approach by applying a key derivation function that changes the parameters of the encrypted data. Instead of storing identical encrypted data for deduplication or keeping original encrypted data for security, the system transforms the data through a deterministic function that produces consistent but non-reversible outputs, achieving both goals.
2Reliability
If each client encrypts files with unique keys before upload, then data security is improved, but storage efficiency deteriorates as duplicate files cannot be deduplicated
Solution Approach 1:
The key derivation function serves multiple purposes: it maintains user data security by being non-reversible, enables server-side deduplication by producing consistent outputs from the same input, and eliminates the need for separate client-side key management. This universal function resolves the contradiction between security and storage efficiency.
3Ease of operation
If clients upload encrypted data without checking for existing copies, then ease of operation is improved, but bandwidth utilization deteriorates due to redundant uploads
Solution Approach 1:
The system performs preliminary transformation of the data through a key derivation function before storage. This preliminary action creates a unique identifier for the data that enables the service to check for existing copies and avoid redundant uploads, reducing bandwidth utilization while maintaining ease of operation for users.
Data Source
AI summary
A cloud storage system supporting user agnostic encryption and deduplication of encrypted files is described. Further the cloud storage system enables users to share a file, a group of files, or an entire file system with other users without a user sending each file to the other users. The cloud storage system further allows a client device to minimize the utilization of bandwidth by determining whether the encrypted data to transfer is already present in the cloud storage system. Further the cloud storage system comprises mechanisms for a client device to inform the cloud storage system of which data is likely to be required in the future so that the cloud storage system can make that data available with less latency one the client device requests the data.