Client-Side Compression for Secure Deduplicated Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems, performing compression at the server after encryption leads to performance degradation, and managing multiple compression algorithms across clients in a deduplication domain is challenging, especially when introducing new or retiring old algorithms, as the server lacks knowledge of the client's data application and compression algorithms used.
Innovation Solution
Implementing client-side compression and encryption methods, where clients chop data into blocks, assign handles, and send commands with hashes and initialization vectors, allowing the server to deduplicate without decryption, while clients manage compression algorithms and key rotations, ensuring compatibility and safe retirement of algorithms across clients.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If compression is performed at the server after encryption, then data security is maintained, but system performance degrades significantly
Solution Approach 1:
The patent divides the compression and encryption tasks between client and server. The client performs compression locally before transmission, while the server handles only encryption and deduplication. This segmentation allows compression to occur outside the secure enclave, avoiding performance degradation while maintaining security through server-side encryption.
Solution Approach 2:
The client performs compression as a preliminary action before data is sent to the server. By compressing data locally first, the amount of data requiring secure transmission and server-side processing is reduced, improving overall system performance while the server still maintains security through encryption.
2Productivity
If the server performs compression, then compression can be accelerated by hardware, but the server lacks knowledge of the application data type needed to select optimal compression algorithms
Solution Approach 1:
The patent applies different compression algorithms at the client based on local knowledge of data types and applications. Each client can select the most appropriate compression algorithm for its specific data, optimizing compression ratios. The server then handles the compressed data uniformly through encryption and deduplication.
3Adaptability or versatility
If multiple clients use different compression algorithms in the same dedup domain, then each client optimizes for its data type, but coordinating algorithm updates and retirements becomes complex
Solution Approach 1:
The patent implements a feedback mechanism where clients report their supported compression algorithms to the server. The server maintains knowledge of which algorithms are active in the system and coordinates updates by notifying clients of algorithm retirements. This feedback loop enables safe algorithm management while allowing multiple compression algorithms to coexist.
Data Source
AI summary
A method of sending blocks of data from a client to be stored at a storage server, wherein for each block compression and encryption is performed at the client, and deduplication is performed at the server. Security is thus enhanced as the block is compressed and encrypted when it is sent over an unsecured network and when it is stored in potentially a third-party backup system. Provisions are made to enable addition of new compression algorithms and for retirement of old compression algorithms, while ensuring that a client would not receive a block which was compressed using an unsupported, e.g., retired, compression algorithm. In some examples a compression algorithm ID is tied to an encryption key version to enable refresh of blocks compressed with old algorithm.