LLM Firewall Response Screening for Confidential Prompt Leakage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) are vulnerable to prompt recovery attacks, which expose confidential information, making it difficult to automatically detect and prevent leakage due to the variety of ways users can phrase such attacks, and human review is infeasible.
Innovation Solution
Implement an LLM query manager with an LLM firewall that logs confidential information, compares LLM responses against this information, and sets a leakage detection signal to block responses containing sensitive data, using string matching algorithms to identify overlaps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If human review of individual user queries is performed to detect prompt recovery attacks, then detection accuracy is improved, but productivity deteriorates due to the large number of queries LLMs process
Solution Approach 1:
The patent introduces an intermediary leakage detection system that sits between the user query and the LLM. This intermediary automatically detects prompt recovery attacks by analyzing queries and responses without requiring human review, thus maintaining high detection accuracy while preserving query processing throughput.
2Reliability
If blocking attacks prior to reaching the LLM is implemented, then reliability is improved by preventing leakage, but device complexity deteriorates due to the need for sophisticated pre-filtering mechanisms
Solution Approach 1:
The patent implements preliminary action by collecting confidential information before the LLM processes queries and storing it for later comparison. This allows the system to detect leakage in real-time without requiring complex pre-filtering mechanisms, thus maintaining reliability while minimizing device complexity.
3Reliability
If instructions are added to queries to prevent security issues, then reliability is improved by reducing harmful outputs, but object-generated harmful factors deteriorate as instructions may be leaked and exposed to attacks
Solution Approach 1:
The patent implements feedback by comparing the LLM response against the collected confidential information and generating a leakage detection signal. This feedback mechanism allows the system to maintain security instructions while detecting and preventing their leakage, thus resolving the contradiction between reliability and harmful factors.
Data Source
AI summary
A method includes receiving, at a server from a user device, a user query to a large language model (LLM), creating an LLM query from the user query and an application context, gathering confidential information from the LLM query, and sending the LLM query to the LLM. The method includes receiving, from the LLM, an LLM response to the LLM query, comparing the LLM response to the confidential information to generate comparison result, and setting a leakage detection signal based on comparison result.