Automated Citations and Trustworthiness Assessment for Generated Text
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) often produce hallucinations, leading to the spread of misinformation, exposure of confidential information, and unrealistic expectations due to insufficient or biased training data, making it difficult to determine the factual accuracy of generated text.
Innovation Solution
A system and method that parses generated text to identify statements, queries remote data sources for sources supporting the information, determines trustworthiness values for each statement, and labels the text based on aggregated trustworthiness, providing users with a user interface to review and modify the text.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated text generation is used, then productivity is improved, but reliability deteriorates due to hallucinations and misinformation
Solution Approach 1:
The patent introduces an intermediary verification system that acts as a mediator between the automated text generation process and the final output. This system queries remote data sources to validate statements, computes trustworthiness scores, and provides fact-checking feedback without requiring manual review of each generated text, thus maintaining productivity while improving reliability
Solution Approach 2:
The patent implements a feedback mechanism where the generated text is automatically evaluated by querying supporting sources and computing trustworthiness metrics. This feedback loop allows the system to identify and flag potentially hallucinated content, enabling continuous improvement of generation quality while maintaining automated workflow efficiency
2Reliability
If comprehensive fact-checking is performed, then reliability is improved, but device complexity increases
Solution Approach 1:
The patent segments the fact-checking process into distinct modular components: statement extraction from generated text, querying remote data sources for verification, computing trustworthiness scores for individual statements, and aggregating scores at the document level. This segmentation allows each component to be independently optimized and managed, reducing overall system complexity while maintaining comprehensive verification
Solution Approach 2:
The system performs self-verification by automatically querying remote data sources and computing trustworthiness metrics without requiring external manual intervention. The automated text generation system essentially checks its own output, reducing the need for additional complex verification infrastructure while improving reliability
3Measurement precision
If statement-by-statement verification is performed, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The patent applies partial verification by focusing computational resources on verifying specific statements within the generated text rather than attempting to verify every single claim exhaustively. The system extracts key statements, queries supporting sources for each, and computes trustworthiness scores, providing sufficiently precise assessment without the time cost of complete exhaustive verification of all possible claims
Data Source
AI summary
In some implementations, the techniques described herein relate to a method including: parsing, by a processor, a generated text to identify statements included within a generated text; querying, by the processor, a remote data source to identify sources for each statement in the statements; determining, by the processor, trustworthiness values for each statement, a trustworthiness value for a given statement determined by computing trustworthiness labels for each source corresponding to a given statement: generating, by the processor, a label for the generated text based on an aggregated trustworthiness of each of the statements; and displaying, by the processor, the generated text and the label within a user interface displayed to a user.