Encrypted Data Similarity Search via Edit Distance Tokens
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face challenges in efficiently searching for similar encrypted data strings without decrypting the entire dataset, which can compromise security and increase computational resources.
Innovation Solution
A method and system for similarity searching encrypted data strings involve calculating edit distances between data strings and reference strings, converting these distances into tokens, encrypting the data, and storing them with associated tokens. During a search, edit distances are determined for a search request, converted into tokens, and compared to stored tokens, with results returned only if they meet a predetermined similarity threshold, allowing selective decryption of matching records.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional databases decrypt large amounts of data to search for similar records, then search functionality is achieved, but security risk increases and computational resources are consumed
Solution Approach 1:
The system pre-computes edit distances between encrypted data strings and reference strings before encryption, converting these distances into token documents that are stored alongside the encrypted data. During search operations, the same edit distance calculations are performed on the search query, and the resulting token documents are compared against the pre-stored tokens. This preliminary preparation of distance metrics allows similarity search to be performed on encrypted data without decryption, eliminating security risks while maintaining search functionality.
2Ease of operation
If traditional databases decrypt large amounts of data to search for similar records, then search functionality is achieved, but computational resources are consumed
Solution Approach 1:
The system extracts the edit distance metric from the encrypted data strings and reference strings, converting these distances into separate token documents. By separating the distance information from the encrypted data itself, the system enables similarity search to operate solely on the extracted token documents during query operations. This extraction eliminates the need to decrypt and process the entire encrypted dataset during search, significantly reducing computational resource consumption while preserving the ability to identify similar records.
3Measurement precision
If edit distances are calculated between all data strings and reference strings, then similarity search accuracy is improved, but device complexity increases
Solution Approach 1:
The system creates token document copies that represent the edit distance relationships between encrypted data strings and reference strings. Instead of storing or processing the actual encrypted data during search operations, the system works with these token document copies that encapsulate the similarity information. This copying approach maintains measurement precision by preserving the exact edit distance metrics while simplifying the search process to involve only token comparisons, thereby reducing device complexity.
Data Source
AI summary
Methods and systems of similarity searching encrypted data strings are disclosed. An exemplary method can include receiving data strings, obtaining a set of reference strings, determining edit distances between each data string and the reference strings, converting each set of edit distances into a document of tokens. A method may further include encrypting the data strings, associating each of the documents with a corresponding data string, and storing the data strings and the associated documents in a memory. A method may continue by receiving a search request, determining a search set of edit distances between the search request and the reference strings, converting the search set of edit distances into a document, comparing the search document with the documents stored in memory to determine which documents are above a similarity threshold compared to the search document, and returning the data strings associated with documents above the similarity threshold.