Perspective-Based Prompt Validation for Generative AI Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems fail to effectively identify inaccurate answers to complex customer queries, leading to a lack of accuracy in generative artificial intelligence responses, particularly in domains involving sensitive or confidential data.
Innovation Solution
A system that modifies prompts by adding additional context or perspectives to queries, using generative AI to generate responses, and evaluates the responses for convergence across multiple models to ensure accuracy, thereby reducing AI hallucinations and increasing the reliability of outputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional systems are used to process customer queries, then system simplicity is maintained, but accuracy of answers to complex queries deteriorates
Solution Approach 1:
The system segments the query processing into multiple independent components: prompt generation module, multiple perspective generation module, convergence evaluation module, and accuracy assessment module. Each module performs a specific function, allowing the complex accuracy improvement process to be broken down into manageable, modular steps that can be independently optimized and maintained.
Solution Approach 2:
The system introduces intermediary elements including multiple AI models that act as mediators between the original query and the final answer, perspective generation mechanisms that mediate between different interpretations of the query, and convergence evaluation processes that mediate between conflicting responses. These intermediaries enable thorough accuracy verification without requiring direct complex processing in a single system.
2Reliability
If multiple AI models and perspectives are used to validate prompts, then accuracy of responses is improved, but computational resources and time required increase
Solution Approach 1:
The system performs preliminary actions by generating multiple perspectives and candidate responses before the final accuracy assessment. This allows potential accuracy issues to be identified early in the process, enabling filtering of unreliable responses before they consume additional computational resources in later verification stages.
Solution Approach 2:
The system implements feedback mechanisms where convergence evaluation results feed back into the prompt selection process. Responses that fail to meet convergence criteria are fed back for re-evaluation or rejection, while convergent responses are confirmed. This feedback loop ensures high reliability by continuously verifying accuracy while avoiding wasteful computation on clearly unreliable responses.
3Measurement precision
If additional context and perspectives are added to prompts, then accuracy threshold is met, but prompt complexity increases
Solution Approach 1:
The system applies local quality by adding context and perspectives selectively rather than uniformly to all prompts. The prompt generation module identifies specific areas where additional context is needed based on the original query characteristics, and only enriches those particular aspects. This targeted approach meets accuracy thresholds while minimizing unnecessary prompt complexity.
Solution Approach 2:
The system makes prompt complexity dynamic by adaptively adjusting the amount and type of additional context added based on the specific query requirements. The perspective generation module dynamically determines which perspectives are relevant for each query, adding only the necessary context to achieve accuracy thresholds rather than applying a fixed complex template to all queries.
Data Source
AI summary
Aspects of this technical solution can receive, via a user interface, a first prompt for a large language model including a first query that references first data, generate one or more second prompts for the large language model based on the first prompt and the first data, each of the second prompts including one or more second data clarifying the first query, generate, by the large language model receiving one or more of the second prompts, one or more responses to the one or more second prompts, select an optimized prompt from among the one or more second prompts, according to a determination that a response to the at least one of the second prompts meets an accuracy threshold, and cause the user interface to present the optimized prompt or a response to the optimized prompt, the large language model to generate the response using the optimized prompt as input.