Operator Fusion in Stream Computing to Reduce Encryption Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional static database systems are not well-suited for real-time processing and analysis of large volumes of streaming data, as they often struggle to efficiently store, index, and analyze continuous data streams, and the encryption process between nodes in stream computing applications negatively impacts performance.
Innovation Solution
The use of operator fusion management in stream computing environments identifies sensitive data for encryption and combines operators into a single processing element, eliminating the need for network encryption, allowing data to be communicated through function calls and memory references within the host node, thereby reducing encryption overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If encryption is applied between operators in stream computing applications, then data security is improved, but processing speed and performance deteriorate
Solution Approach 1:
The patent combines multiple operators that require secure data exchange into a single processing element through operator fusion. This merging eliminates the need for network encryption between these operators, as they now communicate through internal memory references rather than encrypted network channels, thus resolving the contradiction between security and performance
Solution Approach 2:
The patent introduces a flow graph analysis mechanism that identifies which operator connections require encryption and which can be fused. This intermediary analysis layer selectively applies encryption only where necessary while enabling fusion elsewhere, balancing security requirements with performance optimization
2Adaptability or versatility
If operators are distributed across multiple nodes, then system scalability is improved, but communication overhead and latency increase
Solution Approach 1:
The patent fuses multiple operators into single processing elements, eliminating inter-node network communications for these operators. This reduces communication latency and overhead while maintaining the overall distributed architecture's scalability, as the fusion optimization can be applied selectively to specific operator subsets
3Reliability
If encryption is applied to all operator connections, then data security is improved, but device complexity and processing overhead increase
Solution Approach 1:
The patent applies encryption selectively rather than uniformly across all operator connections. By analyzing the flow graph to identify which specific connections require security, the system applies encryption only where necessary while enabling fusion for connections where security is less critical, thus reducing overall system complexity and overhead
Data Source
AI summary
An embodiment includes analyzing data associated with an original flow graph comprising a plurality of operators of a stream computing application, including identifying a secure network connection between a first operator and a second operator that uses encryption. The embodiment fuses the first operator with the second operator such that a first logical function of the first operator is combined with a second logical function of the second operator. The embodiment then generates a modified flow graph as a modification of the original flow graph that combines the first operator and the second operator and lacks encryption between the first operator and the second operator.