Secure Prompt System Using Special Token IDs for Language Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing language model systems lack effective security measures to differentiate between system instructions and external data, making them vulnerable to malicious attacks that exploit this lack of separation.
Innovation Solution
The implementation of a secure prompt system that uses updatable random strings in prompt templates, synchronized with a tokenizer, to securely separate system instructions from user inputs by mapping random strings to special token IDs, thereby preventing malicious manipulation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If detailed and complex system instructions are provided to language models to prevent attacks, then the system attempts to improve security, but language models fundamentally cannot distinguish between system instructions and outside data
Solution Approach 1:
The system separates system instructions from user data by embedding a unique training string as a delimiter within the prompt. This segmentation allows the language model to clearly distinguish between instructions it should follow and user input it should process, resolving the fundamental inability to differentiate between the two.
Solution Approach 2:
A training string acts as an intermediary element that the language model learns to recognize as a boundary marker. This intermediary enables the model to understand where system instructions end and user data begins, without requiring complex instruction structures.
2Reliability
If random strings are used to represent system instructions, then the system attempts to prevent attacks, but language models have difficulty detecting exactly matching strings and malicious users can input their own random strings to fool the model
Solution Approach 1:
The system performs preliminary training of the language model on a specific training string before deployment. This preliminary action ensures the model learns to precisely recognize and respond to the exact training string delimiter, preventing malicious users from fooling the system with similar but different strings.
Solution Approach 2:
The system changes the parameter of the training string from a simple random string to a specially designed sequence that includes both the training string and associated instructions. This parameter change enhances the model's ability to detect and respond to the delimiter accurately.
3Reliability
If long and complex random strings are used to prevent guessing, then security is improved, but language models are weaker at matching them and more resources are required for retraining
Solution Approach 1:
Instead of using long and complex random strings that are difficult for models to match, the system uses a copied, standardized training string that the model learns during training. This approach maintains security while significantly improving matching efficiency and reducing retraining resources.
4Device complexity
If system instructions and outside data are provided on the same channel, then the system structure is simpler, but they become difficult to differentiate by language models making them vulnerable to attacks
Solution Approach 1:
The system extracts the boundary-detection function from the complex instruction-following capability of the language model. By using a trained delimiter string, the boundary detection is handled through pattern recognition rather than requiring the model to inherently understand instruction versus data boundaries.
Data Source
AI summary
Systems and methods provide for a prompt template to include private random strings that are used by a language model to reference specific tokens on which the language model has been trained. A number of random strings may be generated and inserted into a prompt template and assigned special token identifiers (IDs) in a tokenizer. The text data from the prompt template are tokenized to convert the random strings to the assigned special token IDs which are sent to the language model for inferencing. Based on the provided special token IDs, the language model may reference special tokens learned from training and then generates an inference. The random strings remain hidden during their lifetime and may be updated on-demand to ensure security.