Encrypted Email Search Index Segmentation for I/O Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing solutions for searching encrypted emails face inefficiencies in input/output operations, leading to high latency and resource constraints, particularly in cloud-based systems with millions of users, due to the need for storing and updating large indexes, which is not feasible with current technologies.
Innovation Solution
A three-level data structure is proposed, comprising a non-encrypted hash table on the client, an encrypted oblivious access data structure on the server, and a dynamic append-only encrypted hash table, allowing for efficient storage and retrieval of encrypted document IDs with reduced information leakage and improved IO efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If symmetric searchable encryption is used to search encrypted emails, then search functionality is enabled, but input/output efficiency deteriorates and information leakage increases
Solution Approach 1:
The encrypted index is divided into multiple encrypted index blocks distributed across different storage locations. Each block contains a portion of the encrypted index data, allowing parallel access and reducing the I/O bottleneck when performing search operations on encrypted emails.
Solution Approach 2:
The patent implements a hierarchical index structure where encrypted index blocks are organized in nested levels. The top-level index points to intermediate indexes, which in turn point to leaf-level encrypted index blocks containing actual document identifiers. This nested structure enables efficient traversal and reduces the amount of data that must be read from storage during search operations.
2Speed
If encrypted index is stored in memory to achieve fast search, then search speed improves, but resource feasibility deteriorates for cloud systems with millions of users
Solution Approach 1:
The encrypted index for each user is segmented into multiple blocks that can be stored in distributed cloud storage rather than requiring centralized memory. This segmentation allows the system to scale to millions of users by distributing the index data across storage infrastructure, eliminating the memory resource bottleneck.
Solution Approach 2:
The patent transitions from storing the entire encrypted index in memory (single-dimensional constraint) to a multi-dimensional storage architecture where index blocks are distributed across network storage locations. This dimensional shift from memory-centric to storage-centric architecture enables scalability while maintaining search performance through optimized data retrieval paths.
3Quantity of substance
If encrypted index is stored on disk to save memory, then resource feasibility improves, but input/output bound deteriorates
Solution Approach 1:
The encrypted index is segmented into multiple blocks that can be selectively loaded into memory based on search queries. Only the relevant index blocks needed for a particular search operation are retrieved from disk storage, significantly reducing I/O operations compared to reading the entire index.
Solution Approach 2:
The system pre-organizes encrypted index data into structured blocks with metadata that enables efficient location and retrieval. This preliminary organization of index blocks allows the system to quickly identify and load only the necessary portions from disk storage, reducing I/O overhead and improving overall efficiency.
Data Source
AI summary
The present teaching relates to searching encrypted data. In one example, a search request is received for encrypted documents. An encrypted query is generated based on the search request. The encrypted query is sent to a server that stores a first encrypted index and a second encrypted index. The first encrypted index maps encrypted keywords to full blocks each of which has a same size and is fully filled with encrypted document identities (IDs). The second encrypted index maps encrypted keywords to partial blocks each of which has the same size and is partially filled with encrypted document IDs. Based on the encrypted query, one or more encrypted document IDs are determined by searching against both the first encrypted index and the second encrypted index. A search result is generated based on the one or more encrypted document IDs. The search result is provided in response to the search request.