Tokenized Bigram Search for Protected Distributed Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data stored in databases becomes unsearchable after encoding, posing a security risk when decoded, and exact search results are limited due to variations in query terms.
Innovation Solution
Tokenize database entries and bigrams of each data entry, allowing query terms to be parsed into bigrams for matching tokenized database entries without decoding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is encoded/protected in a database, then data security is improved, but data becomes unsearchable
Solution Approach 1:
The patent segments data into individual characters and replaces each character with a corresponding token from token tables. This segmentation allows the system to maintain security through tokenization while enabling search operations by comparing token sequences without exposing the underlying sensitive data.
2Measurement precision
If exact matching is used for search queries, then search precision is improved, but search utility deteriorates due to misspelled or varied query terms
Solution Approach 1:
The patent implements partial matching by comparing token sequences from query terms with token sequences from stored data. Instead of requiring exact matches, the system identifies partial overlaps in token sequences, allowing it to return relevant results even when query terms contain misspellings or variations, thus balancing precision with versatility.
Data Source
AI summary
Data in a database can be protected, for instance by tokenizing the entries of the database using one or more token tables. To enable searching data within the database without first detokenizing the tokenized database entries, bigrams of each data entry can also be tokenized and stored in association with the tokenized data entry. When a query term is received, the query term can be parsed into bigrams, and each bigram can be tokenized. The tokenized query bigrams can be used to query the database, and tokenized database entries corresponding to tokenized bigrams that match the tokenized query bigrams can be identified and returned as search results.