LLM Summary Self-Critique for Hallucination Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) are prone to hallucinations, generating non-factual and imaginative text, which can be risky for many use cases, and existing methods to minimize these hallucinations are computationally expensive and limit scalability.
Innovation Solution
A self-critic approach is used to minimize hallucinations in LLM-generated summaries by grounding the summary on the subject, prompting the LLM to generate an initial summary emphasizing relevant features, asking factual questions, and refining the summary based on positive question answers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If auxiliary context or knowledge is provided to the LLM to minimize hallucinations, then the reliability of generated text is improved, but the computational cost and complexity increase
Solution Approach 1:
The system performs preliminary extraction of factual information from the input text before generating the summary. By identifying and separating factual claims early in the process, the LLM can be guided to base its summary generation on these pre-validated facts rather than generating potentially hallucinated content during the main generation phase.
Solution Approach 2:
The system introduces an intermediary factual information extraction step between the input text and the summary generation. This intermediary component processes the input text to extract verified factual claims, which then serve as constraints or guidance for the LLM during summary generation, reducing hallucinations without requiring the LLM to access external knowledge sources.
2Reliability
If auxiliary context or knowledge is provided to the LLM to minimize hallucinations, then the reliability of generated text is improved, but the scalability is limited
Solution Approach 1:
The system performs preliminary extraction of factual information from the input text before generating the summary. By identifying and separating factual claims early in the process, the LLM can be guided to base its summary generation on these pre-validated facts rather than generating potentially hallucinated content during the main generation phase.
Solution Approach 2:
The system introduces an intermediary factual information extraction step between the input text and the summary generation. This intermediary component processes the input text to extract verified factual claims, which then serve as constraints or guidance for the LLM during summary generation, reducing hallucinations without requiring the LLM to access external knowledge sources.
3Loss of information
If the LLM is prompted to generate an initial summary emphasizing relevant features, then the relevance of the summary is improved, but the risk of hallucinations increases
Solution Approach 1:
The system performs preliminary extraction of factual information from the input text before generating the summary. By identifying and separating factual claims early in the process, the LLM can be guided to base its summary generation on these pre-validated facts rather than generating potentially hallucinated content during the main generation phase.
Solution Approach 2:
The system changes the parameter constraints provided to the LLM by incorporating extracted factual information as guiding parameters. Instead of simply prompting for relevant features, the system provides specific factual claims extracted from the input text as parameters that the summary must adhere to, thereby maintaining relevance while ensuring factual accuracy.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods are directed to minimizing hallucinations in a generated summary. A summary generation system embodied within a server triggers a large language model (LLM) to generate an initial summary for a subject. Based on the initial summary, the server prompts the LLM to generate a list of factual questions about the initial summary. The server then triggers the LLM to answer the list of factual questions without knowledge of the initial summary and using internal knowledge of the LLM. Questions from the list of factual questions that received a positive answer are identified. Based on the questions, the server prompts the LLM to generate a refined summary from the initial summary. The server then generates a user interface that presents the refined summary. Approval of the refined summary triggers generation of a publication using the refined summary.