LLM Ensemble Retrieval for Token-Limited One-to-Many Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Language models face token constraints that limit their ability to perform large computational matching tasks, particularly many-to-one matching tasks, which overwhelm the token limit and result in inefficient and impractical processing.
Innovation Solution
A method and system that utilize a rule-based application to reduce the number of tokens, followed by a sorting language model to identify candidate matching sets, and a matching language model to select the correct match, all within the token limit, thereby overcoming the token constraint and improving computational speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a language model is used to perform large computational matching tasks, then the versatility and cost-effectiveness of language models is improved, but the token constraint limits the model's ability to process large datasets
Solution Approach 1:
The patent segments the matching task into multiple stages: first using a rule-based application to pre-filter and reduce the dataset to a manageable subset, then using the language model to process this reduced subset and generate candidate matching sets, and finally using another language model to select the correct match. This segmentation allows the language model to work with smaller token quantities while still solving the original large-scale matching problem.
Solution Approach 2:
The patent applies preliminary action by using a rule-based application to pre-process and reduce the dataset before it is fed to the language model. This preliminary reduction of tokens to a subset allows the subsequent language model processing to operate within token constraints while still addressing the original large computational task.
2Quantity of substance
If the number of tokens is reduced to within the token limit, then the language model can process the data, but the computational complexity of the matching task increases due to multiple processing stages
Solution Approach 1:
The patent divides the matching process into distinct segments: rule-based filtering stage, candidate generation stage using language models, and final selection stage. This segmentation manages complexity by breaking down the overall task into manageable components, each handling a specific aspect of the matching problem.
Solution Approach 2:
The patent introduces an intermediary approach by using rule-based applications and intermediate language model calls to bridge the gap between the large input dataset and the token-constrained language model. These intermediaries process and transform the data into formats suitable for the language model's token limitations.
3Quantity of substance
If a rule-based application is used to reduce tokens, then the number of tokens processed is reduced, but the accuracy of matching may be compromised by overly aggressive filtering
Solution Approach 1:
The patent applies partial action by using rule-based filtering to reduce tokens to a subset that is sufficient but not exhaustive. This partial reduction maintains enough data for accurate matching while staying within token limits. The subsequent language model processing then completes the matching task on this partially processed data.
Solution Approach 2:
The patent incorporates feedback mechanisms where the language model evaluates candidate matching sets and provides feedback on their validity. This feedback loop allows the system to adjust and refine the matching results, ensuring accuracy even after token reduction by rule-based applications.
Data Source
AI summary
A method of executing a matching language model to perform a many-to-one matching task. The matching task uses a number of tokens that exceeds a token limit of the matching language model. The number of tokens is reduced to a reduced number of tokens by executing a rule-based application on a target entry and a dataset of entries to output a data subset including fewer entries. The reduced number of tokens are reduced within the token limit by executing a sorting language model on the data subset and the target entry to output candidate matching sets. The candidate matching sets are evaluated as to whether a potential match exists with the target entry. The matching language model is executed on the number of candidate matching sets and the target entry to output a selected matching set matching the target entry. The selected matching set is returned.