LLM Output Routing via Rephrasing Confidence

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Large language models often generate semantically inconsistent outputs and fail to understand human intent, leading to uncertainty and undesirable performance, particularly in applications like chatbots, where small perturbations in input can result in different and inconsistent answers.

Innovation Solution

A method that generates test inputs by modifying user inputs and processing them through a rephrasing model, then compares the outputs to determine similarity scores, which are used to calculate a model confidence score. This score is compared to a confidence threshold, routing the user input to either the original model or an alternative, such as a human facilitator, based on whether the threshold is met.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If large language models are used to generate text outputs, then natural language processing capabilities are improved, but output consistency and reliability deteriorate

Engineering Contradiction:

Improvenatural language processing capabilityVSAvoidoutput consistency

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The system performs preliminary actions by generating multiple test inputs through rephrasing before the actual user input is processed. These test inputs are used to evaluate model confidence in advance, allowing the system to determine whether to use the language model or route to alternative resources before generating the final output.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback by comparing multiple test outputs from the language model, calculating similarity scores, and determining model confidence based on this comparison. This feedback mechanism allows the system to assess the reliability of model outputs and make informed routing decisions.

Inventive Principle:

Principle #23Feedback

2Reliability

If model confidence evaluation is performed through multiple test inputs and similarity comparisons, then output reliability is improved, but computational complexity increases

Engineering Contradiction:

Improvemodel confidence accuracyVSAvoidcomputational complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The system applies partial action by generating a limited number of test inputs through rephrasing rather than exhaustive testing. This approach provides sufficient confidence evaluation without the excessive computational burden of complete testing, balancing reliability with complexity.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS20250021468A1Controlling uncertain output by large language models

Publication Date: 2025.01.16 INTUIT INC

AI summary

A method including receiving a user input from a user device. The method also includes generating test inputs including the user input and modified inputs. The user input is processed with a rephrasing model to form the modified inputs. The method also includes executing a test model to generate test outputs, including an original test output and modified test outputs, from processing the test inputs. The method also includes generating similarity scores by performing similarity comparisons among the test outputs. The method also includes determining a model confidence from the similarity scores. The method also includes routing the user input responsive to the model confidence satisfying or failing to satisfy a confidence threshold.