Automated PII Pseudonymization via Regex Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data scrubbing techniques are manual, incomplete, and often render data useless for data scientists, failing to comply with privacy regulations like GDPR while preserving valuable insights.
Innovation Solution
A computer-implemented method that analyzes metadata to classify data fields, uses machine learning and deep learning to identify personal identifying information (PII), and automatically generates regular expressions for pseudonymization, ensuring compliance with privacy laws without losing data fidelity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual data scrubbing techniques are used to protect privacy, then personal identifying information is removed, but data becomes useless for data scientists and productivity decreases
Solution Approach 1:
The patent extracts only the personal identifying information components from data values while preserving the remaining useful data. Regular expressions are used to identify and remove specific PII patterns (names, addresses, phone numbers) from otherwise valuable data records, allowing data scientists to retain and analyze the non-PII portions.
Solution Approach 2:
The patent applies different processing treatments to different parts of the data based on their sensitivity. Rather than anonymizing entire data fields, the system selectively pseudonymizes only the PII portions while leaving other data intact, creating a locally differentiated quality approach where privacy protection is applied precisely where needed.
2Reliability
If comprehensive data anonymization is applied to ensure privacy compliance, then all personal identifying information is removed, but data fidelity and insights are lost
Solution Approach 1:
The system extracts and removes only the specific PII elements that pose privacy risks, rather than removing entire data fields. This selective extraction maintains data fidelity by preserving the structural integrity and useful information content of the original data while eliminating only the identifying portions.
Solution Approach 2:
The patent changes the parameters of data processing by using configurable regular expressions that can be adjusted to match different PII patterns. This allows the system to adapt to various data types and privacy requirements, changing the processing parameters dynamically to maintain both compliance and data utility.
3Reliability
If manual techniques are used to identify and remove PII, then privacy protection is achieved, but the process is time-consuming and incomplete
Solution Approach 1:
The patent replaces manual mechanical review processes with automated computational methods. Regular expressions and pattern-matching algorithms automatically identify PII across large datasets, substituting human effort with machine-based detection that is both faster and more consistent in identifying privacy-sensitive information.
Solution Approach 2:
The system enables self-service automated PII detection and removal without requiring manual intervention. The regular expressions automatically scan and process data, identifying and pseudonymizing PII elements independently, which eliminates the time-consuming manual review process while maintaining comprehensive privacy protection.
4Reliability
If existing data scrubbing methods are applied, then some PII is removed, but the methods are incomplete and fail to comply with regulations like GDPR
Solution Approach 1:
The patent creates a universal regular expression framework that can handle multiple types of PII patterns across different data formats and contexts. The same core methodology (regex-based pattern matching) serves multiple functions: detecting names, addresses, phone numbers, emails, and other identifying information, providing a single comprehensive solution that meets GDPR and other regulatory requirements.
Solution Approach 2:
The system uses configurable parameters within regular expressions that can be adjusted to match different PII patterns and regulatory requirements. By changing the parameters (patterns, sensitivity thresholds, data types), the same fundamental approach can comply with various privacy regulations including GDPR, making the method both complete and adaptable.
Data Source
AI summary
Computer systems, methods and program products for automating pseudonymization of personal identifying information (PII) using machine learning, metadata, and crowdsourcing patterns to identify and replace PII. Machine learning models are trained for classifying known column names or key names for processing, using metadata. Column or key names are classified to be unprocessed, anonymized or pseudonymized by a pseudonymizer without revealing PII or scrubbing data into a useless format. A library of crowdsourced patterns are utilized for matching PII to data values within column or key names and PII is mapped to replacement methods. Feedback from user annotations retrains the algorithms to improve classification accuracy and Deep Learning algorithms automate the identification of PII using regular expression generation to concisely articulate how pseudonymizers search for PII patterns within a data set. PII replacement is mapped consistently across entire data packages and the crowdsourced pattern library is updated with generated regular expressions.