Two-Stage Symbol String Matching for Data Privacy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Organizations face challenges in detecting and preventing the unauthorized disclosure of sensitive information due to the ease of data access and dissemination, especially across digital platforms, which poses significant security risks.
Innovation Solution
A two-stage symbol string matching mechanism is implemented to efficiently detect and evaluate the similarity of content between data sources, allowing for real-time monitoring and prevention of unauthorized data transfers by using a combination of signatures and characterizations to identify and align potential data matches, while minimizing false alarms and retaining data privacy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If comprehensive data monitoring is implemented to detect unauthorized disclosure, then detection accuracy improves, but false alarms increase causing staff distraction
Solution Approach 1:
The patent segments the data monitoring process into two distinct stages: (1) exact match detection using hash-based signatures for precise identification, and (2) similarity detection using character n-gram analysis for approximate matching. This segmentation allows each stage to specialize in its strength while compensating for the other's weaknesses, thereby improving overall detection accuracy while reducing false alarms through progressive filtering.
Solution Approach 2:
The patent introduces an intermediary filtering mechanism that processes detected matches before generating alerts. The system uses configurable similarity thresholds and contextual analysis as intermediaries to evaluate potential matches, distinguishing between genuine security threats and benign similarities. This intermediary layer reduces false alarms by filtering out low-risk matches that would otherwise trigger staff distraction.
2Speed
If real-time monitoring is implemented across large data stores, then detection speed improves, but computational resources increase
Solution Approach 1:
The patent segments the monitoring task into two phases with different computational characteristics: Phase 1 uses efficient hash-based exact matching that requires minimal computational resources for rapid filtering, and Phase 2 applies more computationally intensive character n-gram similarity analysis only to the small subset of candidates that passed Phase 1. This segmentation enables real-time monitoring by ensuring that heavy computation is performed only when necessary.
Solution Approach 2:
The patent applies partial action by implementing a two-stage filtering approach where the first stage performs a quick but incomplete assessment using signatures, and only the most promising candidates proceed to the second stage of detailed similarity analysis. This partial action at the first stage significantly reduces the computational burden of real-time monitoring while maintaining detection effectiveness.
3Measurement precision
If detailed content analysis is performed to detect similarities, then detection precision improves, but data privacy risks increase
Solution Approach 1:
The patent extracts only the essential characteristics needed for similarity detection—specifically, character n-grams and structural features—while discarding or anonymizing the actual content data. By taking out only the necessary fingerprint information rather than analyzing and storing complete document contents, the system achieves high detection precision while minimizing data privacy risks through reduced data retention.
Solution Approach 2:
The patent uses copying by creating simplified representations (signatures and n-gram profiles) of the original data that capture essential similarity characteristics without containing the actual sensitive information. These copies enable precise similarity detection while the original private data remains protected and is not retained in the analysis system.
Data Source
AI summary
A two-stage, very fast symbol string matching mechanism measures the similarity of content between data sources, allowing for efficient detection and evaluation of exact as well as inexact matches. The disclosed approach pairs complementary encoding technique comprising a first fast lookup that identifies regions of possible similarity, and a second, more computationally intensive analysis that evaluates metrics of similarity and delimits copy boundaries. The original content remains private (encrypted) during processing.