Encrypted Query Processing via Database Sharding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data retrieval methods fail to preserve client and server privacy, as they either require encrypting the entire database or providing an index, which are computationally resource-intensive and do not ensure privacy for both parties.
Innovation Solution
The method involves generating a partitioned database with shards and buckets, where client-encrypted queries are processed in batches using server-encrypted data, allowing for efficient query execution while maintaining privacy by encrypting and decrypting identifiers using commutative encryption techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire database is encrypted to protect server privacy, then server privacy is improved, but computational resource consumption increases and query efficiency deteriorates
Solution Approach 1:
The database is partitioned into multiple shards, each containing a subset of the encrypted data. This segmentation allows the server to process only relevant shards for each query rather than the entire database, reducing computational overhead while maintaining full encryption for privacy protection.
2Productivity
If an index is provided to enable efficient querying, then query efficiency is improved, but client privacy deteriorates as query patterns become visible
Solution Approach 1:
Encrypted identifiers serve as an intermediary between the client's query and the server's database. The client encrypts identifiers before sending queries, and the server processes these encrypted identifiers without being able to decipher them, thus maintaining query efficiency through indexed lookups while preserving client privacy.
3Reliability
If database entries are encrypted to protect both client and server privacy, then privacy is improved, but the complexity of data processing increases
Solution Approach 1:
Data is encrypted in advance during database creation and organized into shards with encrypted identifiers before queries are submitted. This preliminary encryption and organization eliminates the need for complex real-time encryption/decryption operations during query processing, reducing processing complexity while maintaining strong privacy protection.
4Productivity
If the database is partitioned into shards and buckets to improve processing efficiency, then query processing speed is improved, but the system complexity increases
Solution Approach 1:
The database is divided into shards based on encrypted identifier ranges, and each shard is further organized into buckets for efficient data retrieval. This hierarchical segmentation enables parallel processing of multiple queries across different shards, improving processing speed while the use of encrypted identifiers as partition keys maintains relative system simplicity.
Data Source
AI summary
Encrypted information retrieval can include generating a database that is partitioned into shards each having a shard identifier, and database entries in each shard that are partitioned into buckets having a bucket identifier. A batch of client-encrypted queries are received. The batch of client-encrypted queries are processed using a set of server-encrypted data stored in a database. The processing includes grouping the client-encrypted queries according to shard identifiers of the client-encrypted queries, executing multiple queries in the group of client-encrypted queries for the shard together in a batch execution process, and generating multiple server-encrypted results to the multiple queries in the group of client-encrypted queries. The multiple server-encrypted results for each shard are transmitted to the client device.