Neural Network Personal Information Detection With Context Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting personal information in documents suffer from high false positive rates and inefficiencies, particularly in unstructured documents, making it difficult for companies to accurately identify and protect sensitive data in compliance with data privacy regulations.
Innovation Solution
A neural network model trained using a collection of documents to detect personal information by analyzing context through a matrix of feature information, including token vectors and grammatical structures, to provide accurate probability indicators of sentence content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Difficulty of detecting and measuring
If keyword searching methods are used to detect personal information, then detection coverage is improved, but false positive rate increases
Solution Approach 1:
The patent transforms the detection approach from simple keyword matching to a probabilistic classification system using neural networks. The system changes the parameter of detection from binary (match/no match) to continuous probability scores, allowing for more nuanced decision-making and threshold optimization to reduce false positives while maintaining coverage
Solution Approach 2:
The patent replaces the mechanical keyword searching system with an intelligent neural network-based classification system. This substitution enables the system to understand context, semantics, and relationships between words, thereby reducing false positives caused by literal keyword matching in unrelated contexts
2Measurement precision
If comprehensive keyword searching is performed across all documents, then detection accuracy is improved, but processing time increases
Solution Approach 1:
The patent implements preliminary actions by pre-training the neural network model on large corpora and creating feature extraction pipelines before actual detection. The system prepares classification models and contextual understanding in advance, so that during actual document processing, the neural network can quickly evaluate documents without performing exhaustive analysis from scratch
Solution Approach 2:
The patent introduces dynamic elements through the neural network's ability to adaptively weigh different features and adjust probability thresholds based on document characteristics. The system can dynamically prioritize certain detection patterns over others depending on the context, enabling faster processing while maintaining accuracy
3Device complexity
If simple keyword matching is used, then system complexity is reduced, but detection reliability deteriorates
Solution Approach 1:
The patent introduces intermediary components including embedding layers that convert words into meaningful vector representations, and attention mechanisms that mediate between different parts of the input text. These intermediaries enable the system to capture contextual relationships without requiring excessively complex architecture, bridging the gap between simplicity and reliability
Data Source
AI summary
A method, a computing device, and a non-transitory machine-readable medium for detecting personal information. Terms that are of interest are extracted from a corpus of raw text that has been extracted from a collection of documents. For each of the terms, a surrounding sentence is extracted to form a target sentence to thereby form a plurality of target sentences. The surrounding sentence includes at least one reference to a data subject. A matrix of feature information is generated for each of the target sentences to form a plurality of matrices. A neural network model is trained, using the matrices as input, to compute an output that indicates a likelihood of a given sentence containing personal information.