Sensitive Data Detection Using Vector Space and POS Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data security systems rely heavily on pattern matching for identifying sensitive data, leading to high false positive readings and data saturation, which can result in true positives being missed due to overly narrow patterns.
Innovation Solution
A method that combines pattern matching with part of speech analysis and vector space modeling to identify sensitive data, using a part of speech score and vector similarity score to determine if a data element contains sensitive information, thereby reducing false positives and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If pattern matching is used exclusively for identifying sensitive data, then the identification process is simple and fast, but the false positive rate becomes very high leading to data saturation
Solution Approach 1:
The system changes the parameters of data identification by moving from exact pattern matching to fuzzy similarity matching based on vector space models. This allows the system to identify sensitive data based on semantic similarity rather than rigid pattern conformity, reducing false positives while maintaining identification efficiency through optimized vector calculations
Solution Approach 2:
The patent introduces vector space models and part-of-speech analysis as intermediary layers between the raw data and the identification process. These intermediaries transform the identification task from direct pattern matching to a multi-stage process involving vector representation, similarity calculation, and contextual analysis, thereby reducing false positives while maintaining productivity
2Measurement precision
If overly narrow patterns are used to reduce false positives, then the false positive rate decreases, but true positives are allowed to pass undetected
Solution Approach 1:
The vector space model serves multiple functions simultaneously: it captures semantic similarity, accommodates variations in data formatting, and enables flexible threshold adjustment. This multi-functionality allows the system to detect true positives across diverse data formats without requiring overly narrow patterns, while the configurable similarity threshold maintains control over false positive rates
Solution Approach 2:
The system employs dynamic threshold adjustment based on part-of-speech scores and vector similarity metrics. Rather than using fixed narrow patterns, the identification threshold adapts based on the contextual relevance score calculated from multiple features, enabling the system to capture true positives while maintaining false positive control through dynamic rather than static criteria
3Device complexity
If pattern matching alone is used for sensitive data identification, then the system is simple to implement, but the accuracy of identification deteriorates
Solution Approach 1:
The identification system is segmented into distinct modular components: vector space model construction, part-of-speech tagging module, similarity calculation engine, and threshold evaluation layer. This segmentation allows each component to be optimized independently while maintaining overall system manageability, achieving high accuracy through specialized sub-systems without proportionally increasing overall complexity
Solution Approach 2:
The patent replaces the mechanical pattern matching system with a computational vector-based system. Instead of relying on rigid string pattern matching algorithms, the system uses vector space representations and similarity calculations, substituting mechanical rule-based processing with more flexible mathematical modeling that achieves superior accuracy while remaining computationally efficient
Data Source
AI summary
The embodiments provide a system and method for detecting sensitive data in one or more documents, which may be housed in a data storage system. More specifically, the embodiments provide a system that can not only identify potentially sensitive data based on pattern matching, but which can also infer context in unstructured domains. Context may be inferred using both part of speech analysis and by analyzing the similarity of potentially sensitive data with key terms using a vector space model.