AI Pipeline for Clinical Text Anonymization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for automated text anonymization, such as those described in GB2485783A and US20190238516, are inadequate for handling protected health information in discrete text files or image files, and obfuscation techniques make it difficult to train AI models or detect errors in anonymized data.
Innovation Solution
A system and method for automated text anonymization using an AI pipeline with symbolic AI components for detecting and masking protected health information, employing named entity recognition and redaction techniques to generate de-identified clinical text, while allowing for error analysis and model training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If obfuscation techniques are used to anonymize text, then privacy protection is improved, but the ability to train AI models and perform error analysis deteriorates
Solution Approach 1:
The patent introduces an intermediary anonymization layer that replaces sensitive entities with placeholder tokens (e.g., <PERSON1>, <DATE1>) rather than complete obfuscation. This intermediary representation maintains the structural and semantic relationships needed for AI model training while preventing access to actual sensitive information, thus resolving the contradiction between privacy protection and model trainability.
Solution Approach 2:
The system changes the parameter of anonymization from complete obfuscation to selective replacement with structured placeholders. By modifying how anonymization is applied (using parameterized tokens that preserve positional and relational information), the system maintains both privacy protection and the utility of data for AI training purposes.
2Reliability
If comprehensive anonymization is applied to all text data, then privacy compliance is improved, but the accuracy of AI model training deteriorates due to loss of contextual information
Solution Approach 1:
The patent applies local quality by differentiating the treatment of different text elements: sensitive entities are anonymized with structured placeholders while non-sensitive contextual information remains intact. This selective approach ensures privacy compliance for sensitive data while preserving the contextual quality needed for accurate AI model training.
Solution Approach 2:
The system segments text into sensitive entities and non-sensitive context, applying different processing rules to each segment. Sensitive entities receive anonymization treatment with placeholder tokens, while surrounding contextual text remains unchanged, thereby maintaining both privacy compliance and training accuracy.
3Measurement precision
If manual review of anonymized data is performed to ensure accuracy, then detection precision is improved, but productivity deteriorates
Solution Approach 1:
The patent implements feedback mechanisms where the anonymization system continuously learns from validation results and adjusts its entity recognition and replacement strategies. This automated feedback loop improves detection precision over time without requiring manual review of each anonymized document, thus maintaining high productivity while enhancing accuracy.
Data Source
AI summary
A system for automated text anonymisation of clinical text, the system including an AI pipeline module to configure symbolic AI pipeline components for detecting protected health information (PHI) in the clinical text; a masking module for masking the detected PHI in the clinical text and generating a de-identified clinical text output file as well as a corresponding label file with de-identified information. The pipeline components may include at least one non-symbolic AI pipeline component or machine learning model.