Selective PII Redaction in GenAI Outputs Against Prompt Attacks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing generative artificial intelligence models are vulnerable to cyberattacks that manipulate inputs to disclose personally identifiable information (PII) or bypass security measures, leading to unauthorized access and data leakage.
Innovation Solution
Implement a system with a proxy and analysis engine to tokenize and classify PII in model outputs, using machine learning classifiers and policies to selectively redact or block inputs and outputs containing PII, and employ remediation engines to handle malicious prompts and outputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If generative AI models are trained on large amounts of data including PII to improve model performance and capabilities, then the model's predictive accuracy and functionality are enhanced, but the model becomes vulnerable to cyberattacks that can manipulate inputs to disclose PII or bypass security measures
Solution Approach 1:
The patent introduces an intermediary PII detection and redaction system positioned between the generative AI model and the user interface. This intermediary component analyzes model outputs for PII content and selectively redacts sensitive information before presentation to users, thereby maintaining model performance while preventing PII disclosure through cyberattacks
Solution Approach 2:
The system performs preliminary detection and redaction of PII in model outputs before the information can be disclosed to users. By proactively scanning and filtering outputs for PII patterns and using classifiers to identify sensitive content, the system prevents potential data leakage before it occurs
2Reliability
If the system implements comprehensive PII detection and redaction mechanisms to prevent data leakage, then security and data protection are enhanced, but the complexity of the system increases due to additional processing steps
Solution Approach 1:
The PII detection and redaction system is segmented into distinct functional components: tokenization module, pattern recognition module, classifier module, and redaction module. Each component performs a specific function in the PII detection pipeline, making the overall system more manageable and maintainable while providing comprehensive security
Solution Approach 2:
The system employs self-service mechanisms where the PII detection and redaction processes automatically operate on model outputs without requiring manual intervention. The classifiers and pattern recognition algorithms autonomously identify and redact PII content, reducing operational complexity while maintaining robust data protection
3Reliability
If the system redacts all PII from model outputs to ensure complete data protection, then data security is maximized, but useful information and model utility are lost
Solution Approach 1:
The system applies local quality by selectively redacting only the specific PII portions of model outputs while preserving the rest of the useful information. Instead of blanket redaction, the system precisely identifies and redacts only the sensitive elements (such as names, addresses, phone numbers) while maintaining the contextual integrity and utility of the remaining content
4Reliability
If the system implements real-time PII detection and redaction processing to maintain security, then data protection is ensured, but the processing time and response speed increase
Solution Approach 1:
The system implements partial action by applying pattern recognition and classification only to portions of the model output that are likely to contain PII, rather than processing every token uniformly. This selective approach reduces overall processing time while maintaining effective PII detection and protection
Data Source
AI summary
An output of a generative artificial intelligence (GenAI) model is received which is responsive to a prompt by a requestor. The output is tokenized to result in a plurality of tokens. These tokens are then used to determine that the output includes at least one string comprising personally identifiable information (PII). This determined can use pattern recognition to identify tokens and sequence of tokens indicative of PII. Thereafter, a classifier is used to assign a PII type to each string in the output comprising PII. It is then determined that at least one of the PII types in the output requires redaction which results in strings having a PII type determined to require redaction to be redacted which, in turn, results in a modified output for transmission to the requester. Related apparatus, systems, techniques and articles are also described.