LLM Retrieval Graphs for Accurate Biomedical Answers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large Language Models (LLMs) face challenges in maintaining accuracy and reliability in rapidly evolving fields like biomedicine due to reliance on outdated datasets and the difficulty in verifying the factual basis of their responses, which is critical in healthcare where precision is essential.
Innovation Solution
A system and method that augments LLMs with Retrieval Augmented Generation (RAG) using a System Graph to build structured data from unstructured knowledge bases, continuously updating and validating information through expert review and clustering to enhance accuracy and timeliness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If LLMs rely on pre-existing datasets for training, then the models can be developed and deployed, but the information becomes outdated in rapidly evolving fields like biomedicine
Solution Approach 1:
The patent introduces an intermediary retrieval system that sits between the LLM and the user query. This retrieval component fetches up-to-date information from external sources and passes it to the LLM, allowing the model to generate responses based on current data without requiring retraining. This resolves the contradiction by maintaining reliability through the LLM while achieving timeliness through the intermediary retrieval mechanism.
Solution Approach 2:
The system dynamically adapts by combining static LLM knowledge with dynamic retrieval of current information. The LLM provides stable, pre-trained capabilities while the retrieval component continuously updates with new data, creating a hybrid system that is both reliable and timely without requiring frequent model retraining.
2Ease of operation
If LLMs generate responses without direct citations, then the output appears credible and fluent, but it becomes difficult to verify the factual basis of the information
Solution Approach 1:
The system implements feedback by retrieving and presenting source citations alongside LLM-generated responses. This allows users to verify the factual basis of the information while maintaining the fluency of the original LLM output. The citation feedback mechanism resolves the contradiction by preserving ease of operation through natural language generation while enabling measurement precision through verifiable source attribution.
3Reliability
If expert review is performed on clustering and summarizations, then the accuracy of information is improved, but the time and resources required for validation increase
Solution Approach 1:
The system applies partial expert review rather than complete review of all LLM outputs. Experts review selected cases, particularly those involving complex or high-stakes information, while other routine queries are processed automatically. This partial action approach maintains high accuracy for critical cases while preserving overall productivity by not requiring expert validation for every response.
Data Source
AI summary
Systems and methods for improving the accuracy of information obtained using a large language model. In one embodiment, this involves augmenting the capabilities of a graph generated from unstructured data with information from an external source using Retrieval Augmented Generation (RAG). In one embodiment, expert knowledge is used to review clustering and cluster summarizations derived from the results of a search over the graph data and information prior to application of RAG to generate additional information to augment the search results.