Machine Learning PII Classification System
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems lack accuracy and scalability in identifying and classifying personal identifiable information (PII) within large datasets, relying on human inspection and requiring frequent rule updates as new data types emerge, and struggle with processing diverse data formats such as structured, semi-structured, and unstructured data.
Innovation Solution
A system utilizing a machine learning module with preprocessing, feature detection, feature extraction, context recognition, and classification modules to automatically identify and classify PII across various data types by converting data into a machine-readable format, employing techniques like linear regression, logistic regression, and BERT for accurate prediction and grouping of PII within structured, semi-structured, and unstructured data sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If rule-based PII detection systems are used, then the system can identify known PII types, but the system requires frequent rule updates as new data types emerge and lacks accuracy at the PII classification level
Solution Approach 1:
The patent replaces the mechanical rule-based detection system with a machine learning-based system. The machine learning model automatically learns PII patterns and classifications from training data, eliminating the need for manual rule creation and updates. The system uses algorithms to detect and classify PII types dynamically, improving accuracy while reducing operational complexity.
Solution Approach 2:
The system changes the operational parameters from static rule-based detection to dynamic machine learning-based detection. The machine learning model continuously adapts its parameters through training on new data, allowing it to automatically adjust to emerging PII types without requiring manual rule updates. This parameter transformation enables the system to maintain high accuracy across diverse and evolving data types.
2Reliability
If human inspection is used to identify PII, then the system can detect PII with human judgment, but the process is not scalable to large corpuses of documents
Solution Approach 1:
The machine learning system performs self-service by automatically detecting and classifying PII without requiring human inspection. The model has been trained to recognize PII patterns autonomously, enabling it to process large volumes of documents independently. This self-service capability maintains reliable detection while achieving high productivity through automated batch processing of document corpuses.
Solution Approach 2:
The system extracts the human inspection function and replaces it with an automated machine learning system. By taking out the manual review process and substituting it with algorithmic detection, the system maintains the reliability of human judgment while achieving the productivity of automated processing. The machine learning model captures human expertise in PII detection and applies it at scale.
3Productivity
If automated systems process large corpuses of documents, then productivity increases, but the system lacks accuracy and scalability in identifying and classifying PII
Solution Approach 1:
The system performs preliminary action by training the machine learning model on comprehensive PII data before deployment. This pre-training phase enables the model to learn accurate PII identification patterns and classification rules in advance. When processing large document corpuses, the pre-trained model maintains high accuracy while achieving high productivity through automated processing. The preliminary training action ensures the system is ready to handle diverse PII types accurately at scale.
Data Source
AI summary
A system for determination and classification of personal identifiable information in a file using machine learning is disclosed. The system includes a processing subsystem which includes a pre-processing module and a machine learning module. The preprocessing module receives a data source including a plurality of structured data, a plurality of semi-structured data, and a plurality of unstructured data from a web page, converting the data source into a machine-readable format. The machine learning module includes a feature detection module detecting personal identifiable information features from a group of a plurality of groups, a feature extraction module extracts the plurality of personal identifiable information features from the group of at least one of a static list and a stream. The context recognition module contemplates a plurality of data source-specific features to recognize the context of personal identifiable information. The classification module predicts the presence of personally identifiable information.