Prompt Blocklist Screening for Generative AI Model Protection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial intelligence models, particularly large language models, are vulnerable to cyberattacks that manipulate or modify inputs to produce unreliable or malicious outputs, leading to data leakage or unauthorized access.
Innovation Solution
Implementing a prompt blocklist derived from a corpus of known malicious prompts to analyze and identify potentially malicious or sensitive content within inputs and outputs of generative AI models, using similarity analysis and embeddings to determine and mitigate such threats.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If similarity analysis using blocklist is implemented to detect malicious prompts, then security against adversarial attacks is improved, but computational resources and analysis time are consumed
Solution Approach 1:
The blocklist is pre-generated from a corpus of known malicious prompts before runtime analysis. This preliminary action stores malicious patterns in advance, enabling faster comparison during actual prompt analysis without performing complex generation operations in real-time, thus reducing computational resources during security detection.
Solution Approach 2:
The patent extracts specific malicious patterns and characteristics from a large corpus of malicious prompts to create a condensed blocklist. This extraction process isolates the essential malicious features (such as specific tokens, phrases, or structural patterns) from the full corpus, enabling efficient comparison while maintaining detection effectiveness.
2Measurement precision
If comprehensive similarity analysis is performed on all prompts, then detection accuracy is improved, but processing speed is reduced
Solution Approach 1:
The patent applies partial action by performing similarity analysis only on relevant portions of prompts using the blocklist, rather than analyzing every aspect of each prompt comprehensively. The blocklist enables targeted comparison of specific malicious patterns, achieving sufficient detection accuracy without the computational overhead of exhaustive analysis of all prompt components.
Data Source
AI summary
The inputs and/or outputs of a generative artificial intelligence model are monitored to determine whether they contain or otherwise elicit undesired behavior by the model such as bypassing security measures, leaking sensitive information, or generating or consuming malicious content. This determination can be used to selectively trigger remediation processes to protect the model from malicious actions. Related apparatus, systems, techniques and articles are also described.