LLM Prompt Segmentation and Knowledge Graph Rules Against Injection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models are vulnerable to prompt injection attacks, which can manipulate their behavior to generate biased or undesirable outputs, posing risks to organizations, particularly in core decision-making systems.
Innovation Solution
A computing platform segments prompt injection requests, analyzes them for new learnings, generates knowledge graphs to determine new rules, and assesses their impact using key performance metrics before implementation, employing continuous knowledge graph analytics to prevent such attacks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If large language models are made autonomous with self-learning abilities, then their versatility and adaptability improve, but their vulnerability to prompt injection attacks increases
Solution Approach 1:
The patent introduces an intermediary system consisting of a prompt injection detection model and knowledge graph that mediates between user inputs and the large language model. This intermediary analyzes incoming prompts for potential injection attacks before they reach the autonomous model, allowing the model to maintain its self-learning capabilities while being protected from malicious inputs through the protective layer of the detection system
2Reliability
If prompt injection detection is implemented using traditional methods, then security improvement is achieved, but the complexity and computational overhead increase
Solution Approach 1:
The patent transforms the detection approach by changing the parameters from traditional signature-based or rule-based methods to a machine learning-based detection model that uses knowledge graphs. This allows the system to detect prompt injection attacks through pattern recognition and semantic analysis rather than rigid rules, reducing complexity while improving reliability
Solution Approach 2:
The detection system is designed to continuously learn and update itself by analyzing new prompt injection patterns and incorporating them into the knowledge graph. This self-updating capability reduces the need for manual system configuration and maintenance, thereby reducing operational complexity while maintaining high security standards
3Measurement precision
If continuous monitoring and analysis of prompt inputs is performed, then detection accuracy improves, but processing time and computational resources increase
Solution Approach 1:
The patent implements preliminary action by pre-processing and segmenting prompt inputs into manageable components before analysis. The system breaks down complex prompts into smaller units and pre-loads relevant knowledge from the knowledge graph, so that when detection is needed, the analysis can proceed more quickly with pre-prepared data structures and indexed information
4Reliability
If knowledge graphs are generated and continuously updated, then detection capability improves, but system complexity and data processing requirements increase
Solution Approach 1:
The patent applies segmentation by dividing the knowledge graph into modular, topic-specific sub-graphs that can be independently managed and updated. Rather than maintaining one large monolithic knowledge graph, the system segments knowledge into manageable domains (e.g., security concepts, prompt patterns, attack vectors), reducing the complexity of graph management while maintaining comprehensive detection capability across all segments
Data Source
AI summary
Aspects of the disclosure relate to using machine-learning large language models to prevent prompt injection attacks to protect enterprise-managed information and resources. In some embodiments, a computing platform may receive a prompt injection request which is segmented for analysis. The segmented prompt injection request may be analyzed to determine if new learnings are required. If new learnings are required, knowledge graphs are generated to determine new rules for the machine-learning large language model to prevent deceptive prompt injection attacks. The generated new rules may be analyzed to determine the impact on the enterprise based on key performance metrics or organizational health factors before approval and implementation.