Question-Answer Dataset Generation for Hallucination Testing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Generative language processing machine learning models often generate hallucinations, which are difficult to detect manually or automatically, limiting their utility to low-risk, low-impact use cases, and existing solutions for preventing hallucinations are impractical for many users.

Innovation Solution

An ensemble of machine learning models is used to generate question-answer pairs from domain-specific source documents, applying confidence and consistency filters to create a dataset for automated hallucination testing, excluding unreliable pairs and ensuring the dataset is factually correct.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual verification is used to detect hallucinations, then accuracy of detection is improved, but time consumption and operational complexity increase significantly

Engineering Contradiction:

Improvehallucination detection accuracyVSAvoidverification time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system uses generative language models to automatically generate question-answer pairs and confidence metrics, enabling the model to self-evaluate its own outputs without requiring external expert verification for every response

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

An automated evaluation system acts as an intermediary between the generative model and final output, using confidence metrics and domain-specific knowledge bases to filter and verify responses before presentation to users

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If automated responses are generated without verification, then productivity is improved, but reliability deteriorates due to hallucinations

Engineering Contradiction:

Improvecontent generation efficiencyVSAvoidinformation accuracy

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system performs preliminary generation of question-answer pairs and confidence metrics before final output delivery, filtering out low-confidence responses in advance to prevent hallucinations from reaching users

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback loops where confidence metrics from domain-specific models are used to evaluate and filter generated content, continuously improving reliability through automated validation against knowledge bases

Inventive Principle:

Principle #23Feedback

3Measurement precision

If domain-specific expertise is required for verification, then measurement precision is improved, but device complexity and operational difficulty increase

Engineering Contradiction:

Improvedomain-specific verification accuracyVSAvoidsystem complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system uses domain-specific language models that are trained copies of expert knowledge, enabling automated verification without requiring actual domain experts to perform manual checks

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent replaces manual expert verification (mechanical human process) with automated machine learning models that process and evaluate domain-specific content, reducing operational complexity while maintaining verification accuracy

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Reliability

If confidence filtering is applied to question-answer pairs, then reliability is improved, but productivity decreases due to additional processing steps

Engineering Contradiction:

Improvedataset qualityVSAvoiddataset generation speed

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The system applies confidence filtering selectively to ensure minimum quality thresholds are met, rather than exhaustive verification of every single question-answer pair, balancing reliability with generation speed

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS20250348710A1Automated generation of a dataset of question-answer pairs for domain-specific hallucination testing of generative language processing machine learning models

Publication Date: 2025.11.13 INTUIT INC

AI summary

Aspects of the present disclosure provide techniques for automated generation of a dataset of question-answer pairs for domain-specific hallucination testing of generative language processing machine learning models. Embodiments include providing a first block of natural language text from a domain-specific source document as an input to each of a plurality of question-answer pair generation models. Embodiments include obtaining one or more confidence metrics for each of the plurality of question-answer pairs generated by each of a plurality of question-answer pair generation models. Embodiments include filtering one or more question-answer pairs based on the one or more confidence metrics generated for the one or more question-answer pairs. Embodiments include generating a dataset for domain-specific hallucination testing of a generative language processing machine learning model based on the filtering.