Spark Query Operators in SGX Enclaves for Secure Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The safety of underlying operators in Spark query processes cannot be guaranteed, as data may be obtained through memory dumps, compromising security during computing operations.
Innovation Solution
A trusted computing-based Spark query method and system that uses the SGX API to re-implement operators, generate execution code, and invoke an enclave trusted environment for secure execution, ensuring the confidentiality and integrity of data through SGX remote authentication and encryption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional Spark query execution is used, then processing speed and scalability are improved, but security of computing operations deteriorates due to memory dump vulnerabilities
Solution Approach 1:
The patent introduces SGX enclave as an intermediary trusted execution environment between the Spark query system and the underlying hardware. The enclave acts as a mediator that provides security guarantees while maintaining the performance of Spark operations by allowing controlled access to sensitive data and operations through authenticated interfaces.
Solution Approach 2:
The patent implements a nested structure where the SGX enclave is embedded within the Spark execution framework. The enclave contains the critical security-sensitive operations, which are themselves nested within the broader Spark distributed computing system, creating layers of protection while maintaining overall system performance.
2Reliability
If SGX enclave trusted environment is invoked for secure execution, then security of computing operations is improved, but system complexity increases
Solution Approach 1:
The patent extracts the security-critical operations from the main Spark execution flow and places them into separate SGX enclave modules. This extraction allows the complex security mechanisms to be isolated and managed independently, reducing the complexity burden on the overall system while maintaining security guarantees.
Solution Approach 2:
The SGX enclave implements self-service mechanisms for security management, including automatic authentication, cryptographic key management, and trust verification. This self-service capability reduces the need for external security management infrastructure, thereby reducing overall system complexity despite the enhanced security features.
Data Source
AI summary
The present disclosure provides a spark query method and system supporting trusted computing. The method comprises receiving a SQL query statement input by a user; performing parsing, logical plan analysis, logical plan optimization and physical plan generation for the SQL query statement; re-implementing an operator in the physical plan by using SGX API, and generating an execution code; invoking an enclave trusted environment to implement trusted computing for the operator, and executing the execution code. Through the present disclosure, safety of data during the calculation is ensured while the query request is processed in parallel through the spark architecture. Moreover, the spark interface provides the user with more available service and is user-friendly.