Contextual Entity Identification to Curb LLM Hallucinations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The complexity of medical records, comprising structured and unstructured data, poses challenges for comprehensive searches, leading to significant latency and processing power requirements in Electronic Medical Record (EMR) systems.
Innovation Solution
Utilizing Large Language Models (LLMs) to process both structured and unstructured text data, generating natural language queries, and applying a time-based approach to prioritize language model use, thereby enhancing the efficiency of clinical applications like patient eligibility determination for trials.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If comprehensive searches are performed through EMR systems containing both structured and unstructured data, then detection accuracy of criteria-satisfying patients is improved, but processing time and computational resources increase significantly
Solution Approach 1:
The patent segments the search process into two distinct phases: (1) a filtering phase using structured data queries to quickly identify candidate patients, and (2) a verification phase using LLMs to evaluate unstructured data for those candidates only. This segmentation avoids applying computationally expensive LLM processing to all patients, thereby reducing overall processing time while maintaining detection accuracy.
Solution Approach 2:
The system performs preliminary filtering using structured data (ICD-10 codes, demographics, vital signs) before applying LLM-based unstructured data analysis. By pre-identifying candidate patients through fast structured queries, the system prepares a reduced subset of records that require intensive LLM processing, thus minimizing total processing time while ensuring no true positives are missed.
2Adaptability or versatility
If LLMs are used to process both structured and unstructured text data, then capability to evaluate natural language criteria is improved, but token consumption and processing costs increase
Solution Approach 1:
The patent extracts and processes only the unstructured data portions (clinical notes, narratives) for candidates identified in the filtering phase, rather than feeding all structured data into LLMs. This extraction approach minimizes token consumption by limiting LLM input to only the necessary unstructured text that requires natural language understanding, while structured data is handled by efficient database queries.
Solution Approach 2:
The system applies LLM processing partially - only to the unstructured data of candidate patients identified through structured filtering, rather than processing all patient records completely. This partial application of LLM capability reduces token consumption while maintaining sufficient versatility for evaluating natural language criteria against the relevant data portions.
3Reliability
If all patient records are processed to determine criteria satisfaction, then completeness of results is improved, but computational resources and processing power increase
Solution Approach 1:
The patent divides the patient population into two groups: (1) candidates identified through structured data filtering who proceed to full LLM-based unstructured data analysis, and (2) patients excluded by the filter who require no further processing. This segmentation ensures result completeness for the candidate group while avoiding wasteful computational resource expenditure on clearly non-qualifying patients.
Solution Approach 2:
The system performs preliminary identification of candidate patients using efficient structured data queries before committing computational resources to intensive LLM processing. This preliminary action ensures that only patients with a reasonable chance of meeting criteria undergo resource-intensive analysis, maintaining result completeness while optimizing computational resource utilization.
Data Source
AI summary
A system for mitigating language-model hallucinations during entity identification can include a server that stores multiple entity records in a database and, upon receiving from a client a request containing selection criteria, retrieves a first record for the candidate entity. The server forms a first natural-language prompt containing questions that embody the criteria plus the first record data, submits the prompt to a language model, and obtains a first answer. If the first answer alone fails to satisfy the criteria, the server retrieves a second record for the same entity, constructs a second prompt repeating the questions and appending the second record, and obtains a second answer from the model. The server then evaluates the aggregate of the first and second answers; when the combined information fulfills the selection criteria, the server issues an alert identifying the entity.