Distributed Secret Management with Sharded Encryption Keys
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Centralized key encryption systems in distributed systems face issues such as server outages leading to cascading failures, scaling challenges, and key rotation problems that can render encrypted data inaccessible, along with risks from insider access and unmonitored key usage.
Innovation Solution
A stateless Crypto-as-a-Service (CaaS) system uses a sharded encryption service with unique Data Encryption Keys (DEKs) encrypted by Transit Encryption Keys (TEKs) stored in cloud service provider object storage, ensuring TEKs are isolated and managed independently, with logging for observability and seamless algorithm updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a centralized key encryption system is used, then key management is simplified, but server outages lead to cascading failures and reduced reliability
Solution Approach 1:
The patent segments the centralized key management system into distributed key shards stored across multiple independent nodes. Each node holds a portion of the key material, and keys are encrypted using multiple independent encryption keys rather than a single centralized key. This segmentation eliminates the single point of failure while maintaining manageable key distribution through the use of threshold cryptography and distributed key generation protocols.
2Speed
If keys are stored centrally for easy access, then key retrieval is fast, but scaling becomes difficult and insider access risks increase
Solution Approach 1:
The patent divides key storage across multiple distributed nodes rather than a single central repository. Each node can independently serve key requests, enabling linear scaling as nodes are added. The system maintains fast key retrieval through parallel key reconstruction from multiple nodes, where the key material is reassembled from distributed shards using threshold cryptography, achieving both scalability and speed.
Solution Approach 2:
The patent implements local key caching at edge nodes and clients to maintain fast key retrieval speeds. Frequently accessed keys are cached locally rather than always fetching from the distributed key management service. This local quality approach ensures rapid key access while the distributed architecture provides scalable backend storage and management capabilities.
3Reliability
If key rotation is implemented to improve security, then forward security is enhanced, but encrypted data may become inaccessible
Solution Approach 1:
The patent implements preliminary action by establishing key versioning and binding encrypted data to specific key versions before rotation occurs. When key rotation is initiated, the system pre-generates new key material and maintains multiple valid key versions simultaneously. Encrypted data is tagged with its associated key version identifier, enabling seamless retrieval and decryption using the appropriate key version regardless of subsequent rotations, thus preventing data inaccessibility.
Solution Approach 2:
The patent creates and maintains copies of key material in multiple versions and locations. When key rotation occurs, old key versions are preserved as copies alongside new keys. The system maintains a history of key versions and their associations with encrypted data, allowing retrieval of appropriate key copies for decryption even after multiple rotations. This copying strategy ensures forward security while maintaining continuous data accessibility.
4Reliability
If TEKs are stored in cloud object storage for isolation, then security is enhanced, but access complexity increases
Solution Approach 1:
The patent introduces a key management service as an intermediary layer between applications and the cloud object storage where TEKs are stored. This service handles the complexity of key retrieval, decryption, and management automatically. Applications interact with the intermediary service using simple API calls, while the service manages the complex operations of fetching encrypted TEKs from cloud storage, decrypting them using distributed key shards, and returning the plaintext keys. This intermediary approach maintains high security through cloud storage isolation while keeping access complexity low for end users.
Data Source
AI summary
A data platform provides for encryption of secrets. During operation, an application of the data platform receives a secret and communicates the secret to an encryption client of the data platform. The encryption client generates an encrypted secret using a Data Encryption Key (DEK) and the secret. The encryption client communicates the DEK to an encryption server of the data platform while retaining the encrypted secret. The encryption server generates an encrypted DEK using a Transit Encryption Key TEK. The encryption server communicates the encrypted DEK to the encryption client and the encryption client generates a binary large object (blob) using the retained encrypted secret and the encrypted DEK. The application stores the blob on a data storage device.


