Prompt Filter Ensemble for Grammar-Based Evasion Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing content filters for large language models (LLMs) are vulnerable to evasive prompts that employ grammatical changes and vacuous phrases to bypass detection, and current machine learning systems are ineffective in addressing these obfuscated prompts.
Innovation Solution
A genetic algorithm generates evasive prompts with grammar errors to evade content filters, and a disambiguation model corrects grammar to recover original instructions, deployed with a prompt classifier to identify malicious or benign prompts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If content filters use traditional machine learning systems to detect malicious prompts, then the filtering process is simple and fast, but the system becomes vulnerable to evasive prompts with grammatical changes and vacuous phrases
Solution Approach 1:
The system performs preliminary grammar correction on incoming prompts before they reach the content filter. By correcting grammatical errors and removing vacuous phrases in advance, the system prepares the prompt in a standardized form that makes it more resistant to evasive variations. This preliminary processing ensures that even if attackers use grammatical changes to bypass filters, the correction step restores the intended meaning before filtering occurs.
Solution Approach 2:
The invention introduces an intermediary component (the grammar correction module) between the user input and the content filter. This intermediary processes the prompt by correcting grammar and removing filler phrases, thereby mediating between the raw input and the filtering mechanism. This intermediary layer enhances the system's ability to handle evasive prompts without requiring changes to the core content filter logic.
2Measurement precision
If content filters strictly block prompts with grammar errors to prevent evasive attacks, then detection accuracy improves, but false positives increase and legitimate prompts with minor errors are blocked
Solution Approach 1:
Instead of blocking prompts with grammar errors, the system performs preliminary grammar correction to restore the intended meaning of the prompt. This correction process distinguishes between deliberate evasive obfuscation and legitimate grammatical mistakes, correcting the former while preserving the latter. The corrected prompt is then evaluated by the content filter, which reduces false positives by understanding the true intent rather than penalizing surface-level grammatical issues.
Solution Approach 2:
Rather than blocking prompts based on the presence of grammar errors (the traditional approach), the invention inverts the strategy by correcting the errors first and then evaluating the corrected content. This inversion allows the system to identify evasive prompts through their correction patterns while giving legitimate prompts with minor errors a fair chance to pass through after correction.
3Reliability
If the system processes every prompt through grammar correction and classification, then evasive prompt detection improves, but processing time and computational resources increase
Solution Approach 1:
The system applies grammar correction and classification selectively rather than uniformly to all prompts. By using heuristics to identify prompts that are likely to be evasive (such as those with significant grammatical anomalies or unusual patterns), the system applies the full processing pipeline only to suspicious cases. This partial action approach maintains high detection accuracy for malicious prompts while reducing unnecessary processing overhead for legitimate, straightforward prompts.
Solution Approach 2:
The system incorporates feedback mechanisms where the classification results and correction outcomes inform future processing decisions. By learning from past interactions and identification patterns, the system can adjust its sensitivity and processing intensity dynamically. This feedback loop allows the system to maintain high reliability in malicious prompt detection while optimizing processing time by reducing redundant corrections on clearly benign prompts.
Data Source
AI summary
A genetic algorithm is implemented to generate prompts that evade content filters of generative artificial intelligence (AI) systems. The genetic algorithm applies grammar operations to mutate candidate prompts, communicates the candidate prompts to generative AI systems, and selects candidate prompts that successfully evade content filters according to corresponding responses. A disambiguation model that corrects grammar in prompts is tested on the selected prompts to determine if grammar is properly corrected. Once tested, the disambiguation model is deployed in an ensemble with a classifier that outputs verdicts for prompts with grammar corrected by the disambiguation model.