Staged PII Detection with Regex-Updated Detector Chains
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer-implemented techniques for detecting personally identifiable information (PII) in large corpora of documents are inefficient and impractical due to high computational costs, making it difficult to comply with regulatory obligations.
Innovation Solution
Implementing a chain of successively more accurate PII detectors where downstream stages inform upstream stages, reducing the need for extensive computing resources by optimizing the detection process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If highly accurate PII detection techniques are applied to large corpora of documents, then detection accuracy is improved, but computational cost becomes prohibitively high
Solution Approach 1:
The patent segments the detection process into multiple stages: a first stage using less computationally intensive techniques (e.g., regex patterns, simple classifiers) to identify candidate PII regions, and a second stage applying more accurate but resource-intensive techniques only to these candidates. This segmentation resolves the contradiction by achieving high overall accuracy while limiting computational cost to only the necessary portions of the document corpus.
2Reliability
If comprehensive PII detection is performed across all documents, then regulatory compliance is improved, but processing time increases significantly
Solution Approach 1:
The patent applies preliminary detection actions in the first stage to identify and flag potential PII occurrences before applying comprehensive analysis. By performing preliminary screening with fast techniques and only applying thorough detection to identified candidates, the system ensures regulatory compliance while minimizing overall processing time across large document corpora.
Data Source
AI summary
Methods, systems, and computer program products for detection of personally identifiable information (PII). A first detector and a second detector are configured to interoperate. The first detector is different from the second detector and the second detector incurs a greater computational cost than the first detector when processing identical content. Content is presented to the first detector so as to implement a first type of PII detection that is based at least in part on regular expression analysis using regular expressions. The content is presented to the second detector. The second detector performs PII detection based on content analysis that is different from the first detector's regular expression analysis. The second detector causes generation of new regular expressions based on the content analysis and the first detector is updated with such new regular expressions. Performance of the first detector is continually improved as new regular expressions are generated.