Phrase Matching via Hash Set Scoring for Data Leakage Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data leakage detection methods, such as phrase matching, face challenges in efficiently identifying sensitive phrases amidst noise words, leading to increased computational complexity as more phrases are added, which affects resource requirements and accuracy.
Innovation Solution
A phrase detection system that generates and compares hashes of phrase terms and content terms, using unigram, bigram, and K-gram hashes, along with scoring mechanisms to determine phrase matches, allowing for efficient detection of sensitive phrases even in noisy content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional phrase matching with state models is used to detect sensitive phrases in noisy content, then phrase detection capability is achieved, but computational complexity increases geometrically as more phrases are added
Solution Approach 1:
The patent segments the phrase matching problem into multiple independent hash computations. Instead of using a monolithic state model that processes entire phrases sequentially, the system divides phrases into individual terms and computes hashes for each term independently. These term hashes are then combined using set operations (union, intersection) to determine phrase matches, thereby segmenting the complex matching task into simpler, parallelizable components.
Solution Approach 2:
The patent introduces hash functions as intermediary representations between the original phrase terms and the matching logic. Each phrase term is transformed into a hash value, and the matching process operates on these hash intermediaries rather than the original text. This intermediary layer enables efficient set-based operations and reduces the computational burden of direct string matching, especially when dealing with noisy content.
2Adaptability or versatility
If traditional phrase matching with state models is used to handle noisy content, then noise tolerance is achieved, but accuracy decreases due to obfuscation by noise words
Solution Approach 1:
The patent applies partial matching by allowing phrase detection even when not all terms are present or when additional noise terms are interspersed. The system computes the union of term hashes for a phrase and then checks if the content's term hashes contain this union as a subset. This partial matching approach tolerates missing terms and extra noise words while still detecting the intended phrase, thereby maintaining accuracy despite noisy content.
3Adaptability or versatility
If more phrases are added to the detection system, then coverage of sensitive information increases, but resource requirements increase geometrically
Solution Approach 1:
The patent merges multiple phrase detection tasks into a unified hash-based framework. Instead of maintaining separate state models for each phrase, the system computes term hashes once and reuses them across multiple phrase patterns. The matching logic merges the hash sets of multiple phrases using set operations, allowing the system to handle many phrases simultaneously with linear rather than geometric resource growth.
Data Source
AI summary
Methods, systems, and apparatus, including computer program products, for generating or using augmentation queries. In one aspect, subject phrases for detection in content are identified. Each phrase has a corresponding cardinality of terms. First hash sets for each of the subject phrases are generated, each first hash set including first hashes of bigram term subsets for each of the phrases. Sub-phrase scores for each of the hashes based on the cardinality of each phrase are assigned. The sub-phrase scores a used to detect the subject phrases in hashes of portions of received content. Other implementations of this aspect include corresponding systems, apparatus, and computer program products.