Split Query Optimization for Encrypted Data Stores
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Remote or cloud storage solutions for data often compromise data security, as sensitive data may be exposed to misuse when stored in multi-tenant databases, and querying encrypted data incurs significant resource overhead due to incompatibility with encryption schemes used at remote databases.
Innovation Solution
A method is introduced to optimize split queries by determining a workload distribution between a remote and a local data store, pushing down compatible operations from the remote to the local data store, and executing the query in a way that maximizes workload at the remote data store to minimize data transfer and decryption overhead, using an operator tree to identify and restructure operations compatible with encryption schemes applied at the remote data store.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is encrypted at the remote data store to prevent unauthorized access, then data security is improved, but query operations become incompatible with the encryption scheme and cannot be performed at the remote data store
Solution Approach 1:
The query is divided into multiple operations, and the system identifies which operations can be pushed down to the encrypted remote data store and which must be performed locally. This segmentation allows compatible operations to execute remotely while maintaining security, and incompatible operations to be handled locally where decryption can occur.
Solution Approach 2:
Different operations within the query are treated differently based on their compatibility with encryption. Compatible operations are executed at the remote data store with encrypted data, while incompatible operations are performed at the local data store after decryption, optimizing both security and query functionality.
2Adaptability or versatility
If incompatible operations are performed at the local data store, then query functionality is maintained, but data transfer and decryption overhead increases
Solution Approach 1:
The system performs preliminary analysis of the query operations to identify which ones can be pushed down to the remote data store before execution. This preliminary action allows compatible operations to be executed remotely, reducing the amount of data that needs to be transferred and decrypted at the local data store.
Solution Approach 2:
Compatible operations are extracted from the query and pushed down to the remote data store, separating them from incompatible operations that must be performed locally. This extraction reduces the workload at the local data store by minimizing data transfer and decryption requirements.
3Loss of energy
If all query operations are performed at the remote data store, then data transfer is minimized, but operations incompatible with encryption schemes cannot be executed
Solution Approach 1:
The query operations are segmented into compatible and incompatible categories. Compatible operations are pushed down to the remote data store for execution on encrypted data, while incompatible operations are reserved for local execution, achieving a balance between minimizing data transfer and maintaining query functionality.
4Reliability
If the query is split between remote and local data stores, then both security and query compatibility are improved, but query execution complexity increases
Solution Approach 1:
The system performs preliminary analysis and generates an optimized execution plan that automatically determines the workload distribution between remote and local data stores. This preliminary action simplifies the execution process by pre-identifying which operations should be performed where, reducing the complexity during actual query execution.
Solution Approach 2:
The system acts as an intermediary that manages the split query execution, coordinating between the remote and local data stores. This intermediary handles the complexity of workload distribution and operation routing, shielding the user from the underlying execution complexity while maintaining security and compatibility.
Data Source
AI summary
Methods and apparatus, including computer program products, are provided for split query optimization. In some example embodiments, a method may include: determining, for a query comprising a plurality of operations, a first workload distribution between a first data store and a second data store, the first workload distribution indicating that a first portion of the query is to be performed at the first data store and a second portion of the query is to be performed at the second data store; and determining, based at least on the first workload distribution, a second workload distribution, the determining of the second workload distribution includes pushing down, to the first portion of the query, a first operation from the second portion of the query such that the first operation is performed at the first data store instead of at the second data store.