Generative AI Context Localization to Reduce Hallucinations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generative AI systems, such as large language models (LLMs), often produce hallucinations due to insufficient training data, improper focus on query context, and ambiguity resolution, leading to inaccurate and misleading responses.
Innovation Solution
A context-localization mechanism that employs automatic prompt engineering and utilizes a local data corpus to refine input queries, incorporating clustering and summarization algorithms to guide the generative AI models, reducing the likelihood of hallucinations and improving prediction accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If LLMs are trained on vast amounts of textual data to improve prediction capability, then the model's general knowledge and language understanding improve, but the model may still produce hallucinations and fail to focus on specific query context
Solution Approach 1:
The patent segments the input query into multiple components: original query, retrieved context documents, and generated prompt. This segmentation allows the system to separately manage general knowledge from specific context information, ensuring the model focuses on relevant context while maintaining prediction accuracy.
Solution Approach 2:
The patent introduces an intermediary retrieval system that acts as a mediator between the user query and the LLM. This intermediary retrieves relevant context documents and constructs enhanced prompts, preventing the LLM from losing context information while maintaining its prediction capabilities.
2Adaptability or versatility
If the LLM considers previous queries and conversations to improve response quality, then conversational capability improves, but the model may lose focus on the current query context and produce inaccurate predictions
Solution Approach 1:
The patent extracts only the necessary context information from previous queries and conversations, retrieving specific relevant documents rather than feeding all conversational history to the LLM. This extraction process maintains conversational capability while preventing context loss and improving query accuracy.
Solution Approach 2:
The system performs preliminary retrieval of relevant context documents before processing the current query. This preliminary action ensures that the LLM receives pre-filtered, relevant information, improving both conversational adaptability and context precision simultaneously.
3Productivity
If no context localization is applied, then the system is simpler and faster, but hallucinations increase and prediction accuracy decreases
Solution Approach 1:
The patent applies preliminary context retrieval and prompt construction before the LLM generates predictions. This preliminary action provides the model with focused context information, reducing hallucinations without significantly impacting response speed, as the retrieval process occurs in advance.
Solution Approach 2:
The system uses automatic prompt engineering that self-adjusts based on the retrieved context and query characteristics. This self-service mechanism dynamically optimizes prompts to reduce hallucinations while maintaining efficient processing, balancing productivity and prediction accuracy.
Data Source
AI summary
A context localization system provides relevant local context to a user query to reduce hallucinations and/or inaccuracies for a generative AI system. In embodiments, a corpus of data may be accessed to provide relevant local context. A user query may be used to obtain relevant portions of the local data, which may then be summarized and combined with the user query to form an engineered prompt that include the relevant local context. The engineered prompt is then provided to a generative AI system. In some embodiments, the engineered prompt may allow for determining user sentiment. Other embodiments may be described and/or claimed.