RAG Content Attribution Using Segment Matching to Cut Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generative models (GMs) often fail to ground responsive content in the underlying data on which they were trained, leading to unnecessary computational resource waste and latency due to large search spaces when determining segment matches.
Innovation Solution
Implement a system that normalizes and segments search result documents and responsive content, comparing segments to constrain the search space by first matching against search result documents, then GM training data if necessary, and applies rules based on source data to modify segments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the system checks segment matches against both search result documents and GM training data, then attribution accuracy is improved, but computational resources and latency increase due to large search space
Solution Approach 1:
The system segments the attribution verification process into two distinct phases: first verifying against search result documents (smaller search space), then only verifying against GM training data if needed. This segmentation reduces computational waste by avoiding unnecessary checks against the entire training dataset when search result documents provide sufficient attribution information.
Solution Approach 2:
The system performs preliminary verification against search result documents before falling back to GM training data. This preliminary action filters out cases where search result documents already provide adequate attribution, preventing unnecessary computational resources from being wasted on redundant verification against the larger training dataset.
2Measurement precision
If the system searches through the entire GM training data to determine segment matches, then attribution completeness is improved, but latency increases
Solution Approach 1:
The verification process is segmented into priority-based stages: first checking search result documents (higher priority, smaller search space), then checking GM training data only when necessary (lower priority, larger search space). This segmentation eliminates unnecessary latency by avoiding exhaustive searches through the entire training dataset when search result documents suffice.
Solution Approach 2:
The system applies partial verification action by only searching GM training data when search result documents fail to provide adequate attribution. This partial action approach avoids excessive computational overhead and latency while maintaining sufficient attribution completeness for most cases.
3Productivity
If the system does not ground responsive content in underlying data, then generation speed is improved, but reliability of content accuracy deteriorates
Solution Approach 1:
The system applies local quality verification by checking segment matches only where relevant - specifically when search result documents or GM training data indicate potential mismatches. This localized verification maintains generation speed by avoiding blanket verification of all content, while improving reliability through targeted accuracy checks where needed.
Data Source
AI summary
Some implementations described herein relate to determining whether to modify segment(s) of responsive content that is generated using a generative model (GM) and based on processing user input and search result document(s) obtained via a retrieval augmented generation (RAG) process in response to receiving the user input. Processor(s) of a system can: receive user input associated with a client device, obtain search result document(s) based on the user input, generate the responsive content using the GM and based on processing the user input and the search result document(s), and determine whether to modify the segment(s) of the responsive content based on whether the segment(s) of the responsive content match the search result document(s). The processor(s) can modify the segment(s) of the responsive content if there is a match to the search result document(s), and can cause modified responsive content to be rendered at the client device.