Generative LLM Prompt Injection Detection with Discriminative Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) are vulnerable to cross-domain prompt injection attacks, which can manipulate their outputs to execute malicious actions, posing significant security risks in task automation systems.
Innovation Solution
Implement a discriminative classification engine, decoupled from the target LLM, to identify and filter out malicious prompts before they reach the target LLM, using a separate LLM-based classifier trained on known misalignment attempts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a generative LLM is used for task automation with flexible natural language processing, then the system can perform a wider range of complex tasks without explicit programming, but the system becomes more vulnerable to prompt injection attacks and manipulation
Solution Approach 1:
The patent introduces a discriminative classifier as an intermediary component between the external data source and the generative LLM. This classifier acts as a security gatekeeper that analyzes incoming prompts for malicious content before they reach the LLM, thereby protecting the system from prompt injection attacks while allowing the LLM to maintain its flexible task automation capabilities
Solution Approach 2:
The system is segmented into distinct functional components: a discriminative classifier for security analysis and a generative LLM for task execution. This segmentation allows each component to specialize in its specific function - the classifier focuses on detecting malicious prompts while the LLM focuses on natural language processing, resolving the contradiction between versatility and security
2Productivity
If the LLM processes external data directly without filtering, then the system maintains simplicity and fast processing, but malicious prompts in external data can manipulate the LLM output
Solution Approach 1:
The discriminative classifier performs preliminary analysis of incoming prompts before they are processed by the generative LLM. By pre-filtering and identifying malicious content in advance, the system can block harmful prompts quickly without requiring complex real-time analysis during LLM processing, thus maintaining processing speed while enhancing security
3Reliability
If a discriminative classification engine is added to filter malicious prompts, then the security against prompt injection is enhanced, but the system complexity increases
Solution Approach 1:
The discriminative classifier serves as a specialized intermediary component that handles security analysis independently. By placing this classifier in a dedicated position in the architecture, the system adds security functionality without requiring complex integration throughout the entire LLM system, thus managing complexity while enhancing reliability
Data Source
AI summary
Systems and methods for detecting and mitigating prompt injection attacks on a generative LLM are disclosed. A deployment scenario is considered, in which the generative LLM supports a task automation function. Prompts are received and interpreted by the generative LLM, and outputs from the generative LLM are used to trigger automation actions. The prompts are constructed based on a combination of user input and external data and are, therefore, vulnerable to prompt injection attacks though manipulation of the external data. To mitigate this risk, a separate discriminative classification, decoupled from the generative LLM, engine is configured to identify malicious prompts, and filter out any malicious prompts before they reach the generative LLM.