BiLSTM Neural Network for PII Detection and Redaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automated methods for removing personally identifiable information (PII) from data are ineffective, leading to significant time and cost issues due to their rigidity and lack of context awareness, which results in both false negatives and false positives, and organizations avoid using them to avoid liability under privacy regulations.
Innovation Solution
The use of bidirectional long short-term memory (BiLSTM) neural network models trained on labeled datasets to detect PII, which are more flexible and context-aware, allowing for multifaceted predictions and improved performance in identifying and removing PII from unstructured data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional automated methods are used to remove PII, then the process is automated and faster, but the accuracy is low leading to false negatives and false positives
Solution Approach 1:
The patent replaces conventional pattern-matching and rule-based automated methods with a machine learning-based system that uses trained models to detect PII. This substitution enables the system to understand context and semantics, significantly improving detection accuracy while maintaining automation and processing speed.
Solution Approach 2:
The patent transforms the detection approach by changing from static pattern matching to dynamic machine learning models that can adapt to different contexts. The system uses multiple detection models with different parameters and thresholds, allowing it to adjust sensitivity based on the specific data being processed, thereby improving accuracy without sacrificing speed.
2Measurement precision
If manual inspection is used to remove PII, then the accuracy is high, but the time and cost are significant
Solution Approach 1:
The patent implements a self-service system where the machine learning model automatically detects and removes PII without requiring manual inspection. The trained models perform context-aware detection autonomously, achieving accuracy comparable to manual review while eliminating the time and cost overhead of human involvement.
Solution Approach 2:
The patent introduces machine learning models as an intermediary between raw data and final processed output. These models act as a smart filter that automatically identifies PII with high accuracy, replacing the need for manual inspection while maintaining detection quality.
3Ease of manufacture
If rigid pattern-based methods are used, then the implementation is simple, but the flexibility and context awareness are lacking
Solution Approach 1:
The patent transitions from static pattern-matching rules to dynamic machine learning models that can adapt to different contexts. The models learn from training data and can adjust their detection behavior based on the specific characteristics of the input data, providing both flexibility and context awareness while remaining implementable through standardized ML pipelines.
Data Source
AI summary
Certain aspects of the present disclosure provide techniques for detecting personally identifiable information, including: receiving a plurality of text strings, each text string of the plurality of text strings associated with a user support session; providing the plurality of text strings to one or more bidirectional long short-term memory (BiLSTM) neural network models; receiving output from the one or more BiLSTM neural network models, the output indicating one or more text data elements in the plurality of text strings comprising predicted personally identifiable information; redacting the one or more text data elements comprising the predicted personally identifiable information from the plurality of text strings to form redacted text strings; and providing, to a data repository, the redacted text strings.