Generative AI Input Filtering via Semantic Paraphrase Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generative AI applications are vulnerable to prompt engineering attacks and model hallucinations, which can lead to undesired responses and the exposure of confidential information.
Innovation Solution
A ML-based system is implemented to identify and filter out input requests that are likely to cause undesired behaviors by using a set of known prompt engineering attacks and hallucination-causing topics, without modifying the prompts. This system generates semantically equivalent paraphrases for harmful input phrases and uses a smaller ML model to compare user requests with these paraphrases to determine similarity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a generative AI model is used to process user requests, then the system can provide intelligent responses, but the system becomes vulnerable to prompt engineering attacks and model hallucinations
Solution Approach 1:
The system performs preliminary filtering of user requests against a database of known harmful prompts and paraphrases before they are sent to the generative AI model. This pre-checking action prevents harmful inputs from reaching the model, thereby maintaining security while preserving the model's intelligent response capabilities.
Solution Approach 2:
The patent introduces an intermediary filtering layer between the user interface and the generative AI model. This intermediary component compares user requests against known harmful patterns and blocks malicious inputs, acting as a protective mediator that allows legitimate requests to pass through while preventing attacks.
2Reliability
If the system filters requests by comparing them against known harmful phrases, then security improves, but the system may block legitimate requests that use similar wording
Solution Approach 1:
The system applies different filtering strictness to different types of requests. It uses semantic analysis to distinguish between harmful phrases that should be blocked and legitimate phrases that may superficially resemble harmful patterns. The filtering mechanism adapts its sensitivity based on the contextual meaning rather than applying uniform blocking.
Solution Approach 2:
The patent employs semantic similarity thresholds and confidence scores as adjustable parameters. By tuning these parameters, the system can optimize the balance between blocking harmful requests and allowing legitimate ones, adapting the filtering sensitivity to minimize false positives while maintaining security.
3Productivity
If the system uses a smaller ML model to filter requests, then processing speed improves, but the filtering accuracy may decrease
Solution Approach 1:
The system creates a simplified representation (copy) of harmful prompt patterns in a database that can be quickly matched against user requests. This copying approach allows rapid filtering without requiring complex real-time analysis, achieving both speed and acceptable accuracy by pre-computing and storing characteristic features of harmful prompts.
Solution Approach 2:
The system performs preliminary filtering using a smaller, faster ML model that compares requests against pre-generated paraphrases of harmful prompts. This pre-prepared database enables rapid matching while the smaller model maintains sufficient accuracy for initial triage, with more sophisticated analysis reserved for borderline cases.
Data Source
AI summary
Techniques for filtering for harmful generative artificial intelligence (AI) results are described. An example of filtering includes receiving a request for an input phrase to be responded to by a generative AI model; comparing to the received input phrase to at least one known harmful input phrase to determine that the received input phrase is to be provided to the generative AI model; providing the received input phrase to the generative AI model; and generating a response by at least in part on an output of the generative AI model.