LLM Security Marker Injection for Prompt Attack Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Large language models (LLMs) are vulnerable to prompt injection attacks, where nefarious users bypass security instructions by phrasing queries in various ways, making it infeasible to automatically detect and block such attacks due to the high volume of queries and diverse phrasing possibilities.

Innovation Solution

Insert a security marker instruction into LLM queries to trigger the injection of a detectable marker in responses, allowing the system to identify and block prompt injection attacks by evaluating the presence of the marker in the response.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If human review of individual user queries is performed to detect prompt injection attacks, then detection accuracy is improved, but processing time and system capacity are reduced due to the large number of queries

Engineering Contradiction:

Improvedetection accuracyVSAvoidprocessing capacity

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The system uses a security marker instruction that causes the LLM to automatically insert a detectable marker into its responses. The LLM essentially serves its own detection needs by incorporating the marker, eliminating the need for external human review of each query while maintaining high detection accuracy through automated marker evaluation

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The security marker acts as an intermediary element between the user query and the LLM response. By inserting this intermediate marker into the query and evaluating its presence in the response, the system creates a simple automated detection mechanism that bridges the gap between high-volume processing requirements and accurate attack detection

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If security instructions are added to LLM queries to prevent prompt injection attacks, then security is improved, but the complexity of query processing and the risk of instruction bypasses increase

Engineering Contradiction:

ImprovesecurityVSAvoidquery processing complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The security marker instruction introduces a distinct, easily detectable marker into the query that creates a visible 'color change' in the response stream. This marker serves as a clear indicator of prompt injection attempts, making security detection as simple as checking for the presence of this distinctive element rather than analyzing complex query structures

Inventive Principle:

Principle #32Color changes

Solution Approach 2:

The detection function is extracted from complex query analysis and simplified to a straightforward marker presence check. By separating the security detection function from the main query processing, the system achieves enhanced security without increasing the complexity of normal query handling

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS12591662B2Security marker injection for large language models

Publication Date: 2026.03.31 INTUIT INC

AI summary

A method includes receiving, at a server from a user device, a user query to a large language model (LLM), creating an LLM query from the user query, inserting an security marker instruction into the LLM query to trigger an injection of a security marker, and sending the LLM query to the LLM. The method further includes receiving, from the LLM, an LLM response to the LLM query, evaluating the LLM response to detect whether the security marker is present in the LLM response, and setting a prompt injection signal based on whether the security marker is present in the LLM response.