Unicode Prompt Obfuscation Detection for GenAI Security
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for protecting large language models and other generative AI systems from cyberattacks are ineffective against attacks that obfuscate prompts using unicode, such as unicode font obfuscation, invisible unicode injections, and GNOY attacks.
Innovation Solution
A system that receives prompts for a generative AI model, tokenizes them, identifies and removes repeating sequences, and then determines whether ingestion by the AI model will result in undesired behavior. If not, the modified prompt is passed to the model; otherwise, remediation actions are initiated.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If unicode obfuscation is used in prompts, then the attack can bypass existing security measures, but the model becomes vulnerable to manipulation
Solution Approach 1:
The system performs preliminary analysis of prompts before they are processed by the GenAI model. By detecting unicode obfuscation patterns and repeating sequences in advance, the system can prevent malicious prompts from reaching the model, thereby maintaining reliability without requiring the model to be modified
Solution Approach 2:
An intermediary security layer is introduced between the user input and the GenAI model. This intermediary component analyzes prompts for unicode obfuscation and repeating sequences, acting as a filter that blocks malicious inputs while allowing legitimate requests to pass through to the model
2Difficulty of detecting and measuring
If tokenization and sequence removal are applied to prompts, then attack detection capability is improved, but processing complexity increases
Solution Approach 1:
The prompt processing is segmented into distinct stages: tokenization, repeated sequence identification, sequence removal, and security analysis. This segmentation allows each stage to be optimized independently and makes the overall complex process more manageable and maintainable
Solution Approach 2:
The system extracts and removes repeating sequences from prompts before security analysis. By taking out the redundant unicode characters that form repeating patterns, the system simplifies the prompt structure while preserving the core semantic meaning, making attack detection more effective
3Reliability
If multiple remediation actions are implemented, then security coverage is improved, but system response time increases
Solution Approach 1:
The system implements partial remediation actions based on the severity and type of detected attack. Not all prompts require the full suite of remediation actions, allowing the system to balance security coverage with processing speed by applying only necessary measures to each specific case
Data Source
AI summary
A prompt for a generative artificial intelligence (GenAI) model which contains unicode is received. The prompt is then tokenized to result in a plurality of tokens. Token forming part of a repeating sequence are identified and then removed to result in a modified set of tokens. The modified set of tokens are subsequently detokenized to result in a modified prompt. It is then determined, whether ingestion of the modified prompt by the GenAI model will result in the GenAI model behaving in an undesired manner. The modified prompt is passed to the GenAI model when it is determined that ingestion of the modified prompt will not result in the GenAI model behaving in an undesired manner. Otherwise, at least one remediation action is initiated when it is determined that ingestion of the modified prompt by the GenAI model will result in the GenAI model behaving in an undesired manner.