Document-Based LLM Response Verification With Structured Citations

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing AI/ML models face challenges in multitenancy contexts due to ineffective data access, hallucination, and prompt injection attacks, leading to inaccurate and malicious outputs.

Innovation Solution

Implementing conversational LLM-based user tenant orchestration, confidence enhancement through structured citations and verification, and adverse input mitigation using example pairs to improve accuracy and security of AI/ML model responses.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If LLMs are used to generate responses from data items, then information access efficiency is improved, but hallucination and inaccurate outputs occur

Engineering Contradiction:

Improveinformation access efficiencyVSAvoidoutput accuracy

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system implements feedback by having the LLM verify its own generated responses against the original data items. The response is checked to ensure it is supported by the retrieved data, creating a self-correction mechanism that reduces hallucination while maintaining efficient information access.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system introduces an intermediary verification step between data retrieval and final response generation. The LLM acts as a mediator that not only generates responses but also validates them against source data, ensuring accuracy without sacrificing the efficiency of direct LLM-based access.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Ease of operation

If LLMs process user inputs directly, then ease of operation is improved, but susceptibility to prompt injection attacks increases

Engineering Contradiction:

Improveuser interface simplicityVSAvoidprompt injection vulnerability

Core Design Contradiction:

Ease of operationVSObject-affected harmful factors

Solution Approach 1:

The system performs preliminary action by validating and sanitizing user inputs before they reach the LLM. The orchestrator checks for potential prompt injection patterns and ensures inputs are safe to process, maintaining ease of operation while preventing malicious attacks.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The orchestrator serves as an intermediary layer between the user interface and the LLM. This mediator validates user inputs, filters malicious content, and ensures safe processing while preserving the simplicity of the natural language interface for legitimate users.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Reliability

If multiple LLMs are coordinated for verification, then output accuracy is improved, but system complexity increases

Engineering Contradiction:

Improveresponse verification accuracyVSAvoidorchestration system complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The system segments the verification process into distinct modular components: data retrieval, response generation, and verification steps. Each LLM performs a specific function in this segmented workflow, improving accuracy through specialization while managing complexity through clear separation of concerns.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The orchestrator is designed as a universal component that can coordinate multiple LLMs with different functions (retrieval, generation, verification). This multi-functional orchestrator manages the complexity of coordinating multiple models while providing a unified interface and standardized verification process.

Inventive Principle:

Principle #6Universality (Multi-functionality)

4Adaptability or versatility

If LLMs access user-specific data in multitenancy contexts, then customization is improved, but data access security challenges increase

Engineering Contradiction:

Improveuser-specific customizationVSAvoiddata access security

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The system applies local quality by implementing tenant-specific data access controls and permissions. Each tenant's LLM interactions are confined to their own data space with appropriate access rights, enabling customization for each user while maintaining security through localized data isolation and permission management.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS20250321991A1Confidence enhancement for responses by document-based large language models

Publication Date: 2025.10.16 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

Systems and methods are provided for implementing confidence enhancement for responses by document-based large language models (“LLMs”) or other AI/ML systems. A first prompt is generated based on data items that are previously received or accessed. The first prompt is used by a first LLM or AI/ML system to extract requested information from the data items. One or more citations are generated and presented within a structured object together with a representation of the extracted information, in some cases, as output from a second LLM or AI/ML system. In some cases, the citations and/or the representation may be verified by a third LLM or AI/ML system, and reliability indicators may be generated for the citations and/or the representation based on determined accuracy of the citations and/or the representation. In this manner, the common issue of hallucinations may be mitigated.