Statement-Level Answer Attribution for Long-Form Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional answer attribution techniques fail to efficiently attribute long form, abstractive answers to multiple distinct portions of an information source, requiring manual and time-consuming verification at a paragraph-level granularity, and lack efficiency in attributing answers to sentences or paragraphs.
Innovation Solution
An answer attribution system decomposes answers into statements using a text decomposition model and attributes these statements to corresponding sentences in a document using a natural language inference model, employing a greedy sentence selection algorithm to increase attribution scores, and visualizes the attribution with document annotations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional answer attribution techniques are used, then the system can provide answer attribution, but the verification process becomes manual and time-consuming at paragraph-level granularity
Solution Approach 1:
The patent segments the answer into multiple statements and attributes each statement to specific sentences in the document, rather than treating the entire answer as a single unit. This segmentation enables automated sentence-level attribution, transforming a manual paragraph-level verification process into an automated statement-level matching process, thereby significantly reducing verification time and improving productivity
2Measurement precision
If automated paragraph-level attribution is used, then the system can attribute answers to documents, but it fails to efficiently attribute long form abstractive answers to multiple distinct portions
Solution Approach 1:
The system decomposes the answer into multiple atomic statements and then attributes each statement to specific sentences in the document. This segmentation approach enables both fine-grained attribution (at the sentence level) and efficient processing by treating each statement independently, resolving the contradiction between precision and productivity
Solution Approach 2:
The patent introduces an intermediary representation layer where answers are first decomposed into statements, then matched to document sentences through automated inference. This intermediary statement-level representation enables precise attribution to multiple distinct portions while maintaining computational efficiency through standardized matching procedures
3Reliability
If manual verification at paragraph-level is performed, then users can verify answer support, but the process is time-consuming and lacks efficiency
Solution Approach 1:
The system performs self-service verification by automatically attributing statements to document sentences using natural language inference models. The automated process generates attribution scores and identifies supporting sentences without requiring manual intervention, maintaining verification reliability while eliminating the time-consuming manual verification process
Solution Approach 2:
The patent replaces the mechanical manual verification process with an automated computational system using natural language inference models. The system automatically compares answer statements with document sentences, calculates attribution scores, and identifies supporting evidence, substituting human manual verification with automated AI-based inference to maintain reliability while reducing time consumption
Data Source
AI summary
In accordance with the described techniques, a processing device receives one or more documents and one or more paragraphs formulated from content of the one or more documents. Using a text decomposition model, the processing device decomposes the one or more paragraphs into a plurality of statements. Using a natural language inference model, the processing device attributes a statement of the plurality of statements to one or more sentences of the one or more documents. Further, the processing device generates one or more annotated documents including at least one visual indication associating the statement with the one or more sentences.