Dual-LLM Prompt Screening Against Injection Attacks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large Language Models (LLMs) are vulnerable to prompt injection attacks, which manipulate their responses to produce unintended and potentially harmful outputs, compromising security and confidentiality, especially in critical systems like autonomous vehicles, healthcare, and finance.
Innovation Solution
Implement a dual LLM system with a Quarantined LLM (Q-LLM) and a Privileged LLM (P-LLM) separated by a controller, where the Q-LLM uses a probabilistic classifier to score prompts for maliciousness, and a parser to isolate control characters, combined with cryptographic techniques to ensure secure communication and compartmentalized access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If LLMs are integrated into critical systems to enhance capabilities and applications, then productivity and functionality are improved, but security vulnerabilities and susceptibility to prompt injection attacks increase
Solution Approach 1:
The system is divided into three distinct components: a Quarantined LLM that processes untrusted content without access to sensitive tools or data, a Privileged LLM that has access to sensitive data and tools but does not directly process user input, and a controller that manages communication between them. This segmentation ensures that even if the Quarantined LLM is compromised through prompt injection, the attacker cannot directly access sensitive resources protected by the Privileged LLM.
Solution Approach 2:
The controller acts as an intermediary between the Quarantined LLM and the Privileged LLM. It receives prompts from the Quarantined LLM, validates them, and only forwards safe prompts to the Privileged LLM. This intermediary layer prevents malicious prompts from reaching the Privileged LLM, thereby protecting sensitive data and tools from compromise.
2Reliability
If LLM compartmentalization is implemented to improve security, then security is improved, but device complexity and implementation difficulty increase
Solution Approach 1:
The system divides the LLM functionality into separate modules (Quarantined LLM, Privileged LLM, and controller), each with clearly defined responsibilities. The Quarantined LLM handles untrusted input, the controller validates and mediates communication, and the Privileged LLM manages sensitive operations. This modular segmentation makes the system more manageable despite increased complexity, as each component can be independently developed, tested, and maintained.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure is directed to systems and methods for enhancing the security of Large Language Models (LLMs). The method includes receiving a prompt at a first LLM, wherein: the first LLM is deployed on a first server; and the first LLM comprises a probabilistic classifier; using the first LLM to generate a score for the prompt, wherein the score is indicative of the risk that the prompt is malicious; and determining, based on the score, whether to forward the prompt to a second LLM or to reject the prompt.