Two-Phase Meta Instruction for Secure LLM Prompt Checking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI-powered natural language processing systems face challenges in maintaining response integrity and accuracy when user-generated prompts elicit inappropriate responses, leading to potential malicious request execution and system abuse, particularly with large language models (LLMs), and the addition of meta-instructions (MIs) can further decrease accuracy.
Innovation Solution
A two-phase meta-instruction (2PMI) mechanism is introduced, which includes a dedicated LLM call to evaluate user prompts with MIs without affecting response accuracy, using a 'honey pot' LLM to detect malicious intent and ensuring accurate output through a two-phase process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If meta instructions are added to LLM prompts to prevent prompt injection attacks, then system security is improved, but the accuracy of the LLM output decreases
Solution Approach 1:
The patent divides the LLM processing into two separate phases: a detection phase where a dedicated LLM evaluates prompts with meta instructions to detect malicious intent, and a production phase where the main LLM generates responses without meta instructions. This segmentation allows security checking without compromising the accuracy of the final output.
Solution Approach 2:
The patent introduces a dedicated detection LLM as an intermediary component that handles security evaluation separately from the production LLM. This intermediary processes prompts with meta instructions to detect attacks, then allows the production LLM to generate accurate responses without the interference of meta instructions.
2Measurement precision
If a dedicated LLM call is used to evaluate prompts with meta instructions, then malicious intent detection is improved, but system complexity increases
Solution Approach 1:
The dedicated detection LLM serves multiple functions: it evaluates prompts for malicious intent, determines whether to block requests, and maintains a threat database. This multi-functionality justifies the additional component by consolidating security-related tasks into a single versatile system element.
Solution Approach 2:
The system performs preliminary evaluation of prompts using the detection LLM before submitting them to the production LLM. This preliminary action identifies and blocks malicious prompts in advance, preventing them from reaching the main system and justifying the additional processing step.
Data Source
AI summary
At least one processor can receive a large language model (LLM) prompt and generate an augmented LLM prompt, the generating comprising adding a meta instruction to the LLM prompt. The at least one processor can send the augmented LLM prompt to the at least one LLM and receiving a check response from the at least one LLM in return. The at least one processor can send the LLM prompt to at least one LLM and receiving a production response from the at least one LLM in return. The at least one processor can determine whether the check response complies with the meta instruction determine a reply according to whether the check response complies with the meta instruction, and cause display of the reply.