Contextual Entity Selection With Time-Based LLM Token Conservation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The complexity of structured and unstructured data in medical records complicates comprehensive searches, leading to significant latency and processing power requirements in Electronic Medical Record systems.
Innovation Solution
Utilizing a language model to process both structured and unstructured data, generating natural language queries, and applying a time-based approach to prioritize language model use and conserve token consumption for efficient processing of large datasets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If comprehensive searches are performed through EMR systems to evaluate natural language criteria against patient records, then detection accuracy of criteria-satisfying data is improved, but processing latency and computational power requirements increase significantly
Solution Approach 1:
The patent segments the comprehensive search process into multiple stages: first filtering records using structured data queries, then applying language model evaluation only to filtered subsets of records that potentially satisfy criteria. This segmentation reduces the number of records requiring full language model processing while maintaining detection accuracy.
Solution Approach 2:
The system performs preliminary filtering of patient records using structured data queries and basic text matching before submitting records to the language model for comprehensive criteria evaluation. This preliminary action eliminates obviously non-matching records, reducing the workload for subsequent detailed analysis.
2Measurement precision
If language models are applied to process all patient records to evaluate natural language criteria, then detection accuracy is improved, but token consumption and processing costs increase
Solution Approach 1:
The patent applies different processing qualities to different records: high-quality language model analysis is applied only to records that pass initial filtering stages and show potential for satisfying criteria, while other records receive simpler processing or are excluded from language model evaluation entirely.
Solution Approach 2:
The system performs partial action by evaluating only a subset of records through the language model rather than all records. Records are selectively processed based on their relevance score from preliminary filtering, avoiding token consumption on records that clearly do not meet criteria.
3Reliability
If all patient records are processed to identify entities satisfying clinical trial criteria, then completeness of results is improved, but processing time and computational resources increase
Solution Approach 1:
The patent implements periodic action through iterative processing: records are processed in batches or cycles, with results from each cycle informing the selection of records for the next cycle. This allows the system to maintain completeness while managing computational load through controlled iteration.
Solution Approach 2:
The system uses feedback from preliminary filtering results to guide subsequent processing. Records that meet certain thresholds from initial queries are selected for language model evaluation, creating a feedback loop that optimizes the balance between completeness and processing efficiency.
Data Source
AI summary
A system for conserving token use with a language model can include a server configured to store a plurality of records in a database; receive, from a client device, a request comprising a set of criteria to identify whether the entity satisfies the set of criteria; generate a first natural language query comprising a first record selected based on a first timestamp; execute a language model the first natural language query to generate a first result; generate a second natural language query comprising a second record selected based at least on the second record corresponding to a second timestamp; execute the language model using the second natural language query to generate a second result; determine the set of criteria is satisfied based at least on the second result; and communicate a message identifying the entity to the client device.