Encryption Key Management in Stream Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In stream processing service environments with a mixed infrastructure, protecting sensitive data across multiple hosts or providers is challenging due to the complexity of encrypting and decrypting data streams, as existing methods fail to securely manage encryption keys and ensure access is restricted to only necessary processing elements.
Innovation Solution
A system and method where processing elements are determined to access data streams, and encryption keys are generated per stream per processing element based on security levels, managed by a security manager, and accessed through an access control list, ensuring secure encryption and decryption across multiple sites.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If encryption is used to protect sensitive data in mixed infrastructure, then data security is improved, but key management complexity increases
Solution Approach 1:
The system segments key management by creating separate key stores at each host level, isolating encryption keys to specific processing elements. Each processing element has its own authenticated access to keys, preventing centralized key management complexity while maintaining security across the distributed mixed infrastructure.
Solution Approach 2:
The patent introduces host-level security managers as intermediaries between processing elements and encryption keys. These security managers authenticate processing elements and provide keys without requiring direct access to other hosts' key stores, simplifying key management in the mixed infrastructure environment.
2Speed
If data is streamed across multiple hosts in real-time, then processing speed is improved, but data protection difficulty increases
Solution Approach 1:
The system performs preliminary authentication of processing elements before data streaming begins. Security managers pre-authenticate processing elements and establish key access rights in advance, allowing real-time data streaming with encryption without adding authentication overhead during the actual processing speed-critical operations.
3Ease of operation
If decryption keys are made accessible to processing elements, then data processing functionality is improved, but security control is worsened
Solution Approach 1:
The patent implements local quality by providing each processing element with decryption keys only for the specific data streams it is authorized to process. Keys are stored locally in host-level key stores with access restricted to authenticated processing elements, ensuring that security control is maintained while processing functionality is enabled.
Data Source
AI summary
A system and method for protecting streams in a mixed infrastructure includes determining processing elements that are to access a data stream in a stream processing environment and determining a security level for each processing element. Keys are generated per stream per processing element in accordance with the security level. The keys are associated with processing elements in an access control list in a location accessible by producing and consuming processing elements. The stream is decrypted for processing using keys released upon authenticating processing elements in accordance with the access control list. At security boundaries, the stream is re-encrypted in accordance with a next processing element.