AI Response Generation with Extractive QA Grounding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI models struggle to generate accurate responses to user queries from unstructured data sources due to the lack of a solid basis for the responses, a phenomenon known as "hallucination," especially when information is dispersed across multiple documents and evolves over time.
Innovation Solution
A processor-implemented method using artificial intelligence models to generate responses by extracting text from unstructured data, anonymizing it, determining relevant text extracts using an extractive QA model, and generating a custom context for a large language model to provide grounded responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a large language model is used to generate responses from unstructured data, then the ability to answer queries is improved, but the reliability of responses deteriorates due to hallucination
Solution Approach 1:
The system segments the response generation process into multiple independent components: an extractive QA model that identifies ground truth text extracts, a contextual information retrieval module that gathers supporting evidence, and a large language model that generates the final response. This segmentation allows each component to specialize and provides verifiable intermediate outputs, reducing hallucination while maintaining query response capability.
Solution Approach 2:
The patent introduces text extracts from the extractive QA model as an intermediary between the unstructured data and the large language model. These text extracts serve as verified ground truth that constrains and guides the LLM's response generation, acting as a mediator that prevents the LLM from generating unsupported hallucinated content while still enabling natural language responses.
2Reliability
If text extracts are used to provide basis for responses, then the reliability of responses is improved, but the complexity of the system increases
Solution Approach 1:
The extractive QA model serves multiple functions simultaneously: it extracts ground truth text extracts for reliability, retrieves contextual information for completeness, and provides training data for the large language model. This multi-functionality reduces the need for separate dedicated components, thereby limiting the increase in system complexity while maintaining response reliability.
Solution Approach 2:
The system performs preliminary extraction of text extracts and contextual information before the large language model generates responses. By pre-processing and organizing the unstructured data into structured text extracts with associated contexts in advance, the system reduces the computational burden during query response and simplifies the overall architecture despite the added reliability layer.
3Loss of information
If information is extracted from multiple documents, then the completeness of information is improved, but the difficulty of processing increases
Solution Approach 1:
The extractive QA model selectively extracts only the relevant text extracts and contextual information needed to answer each specific query from the multiple documents, rather than processing or analyzing all document content uniformly. This targeted extraction approach maintains information completeness by capturing all necessary evidence while significantly reducing processing difficulty by focusing computational resources only on relevant portions.
Data Source
AI summary
Generating responses to queries with text extracts from unstructured data using AI models includes (i) extracting text from unstructured data sources to create machine-searchable documents, (ii) replacing PII and PHI with entity types and attributes, (iii) determining text extracts that indicate criteria, (iv) using a small-scale ML model to perform text searches and find conceptually associated text strings, (v) generating a custom context for a large language model (LLM), (vi) prompting the LLM to generate a response, (vii) combining the response with an extractive QA model to obtain relevant text extracts as response basis, and (viii) providing system-generated recommendations for next best actions based on responses that produce the most optimal outcomes in historical input documents for manually selected or automatically recommended resolution paths.