LLM Entity Classification With Two-Phase Text Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) face challenges due to slow inference times, limited context windows, and hardware limitations, making them impractical for certain applications, especially on less powerful hardware.
Innovation Solution
Fine-tuning LLMs for specific entity recognition tasks using smaller parameter sets and applying them in a two-phase classification process with high recall and precision phases, along with post-processing to filter redundant results, allows efficient classification of unstructured text on less powerful hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large language models are used for entity recognition, then classification accuracy is improved, but inference time increases and hardware requirements increase
Solution Approach 1:
The patent segments the entity recognition process into two distinct phases: a high recall phase that identifies potential entities broadly, and a high precision phase that verifies and refines those identifications. This segmentation allows the system to achieve accurate classification while reducing overall inference time by processing documents in staged batches rather than requiring full LLM processing of entire documents.
Solution Approach 2:
The patent applies partial action by using the LLM only for specific verification tasks in the high precision phase rather than processing entire documents through the full LLM pipeline. The system processes only relevant portions of text that require LLM validation, reducing unnecessary computational overhead while maintaining classification accuracy.
2Measurement precision
If large language models with large parameter sets are used, then classification accuracy is improved, but hardware requirements and computational resources increase
Solution Approach 1:
The patent divides the classification workload between simpler preprocessing components (entity extraction rules, batching logic) and the LLM verification stage. This segmentation allows the system to achieve high accuracy using LLMs only where necessary, while relying on less resource-intensive methods for initial processing, thereby reducing overall hardware requirements.
Solution Approach 2:
The patent employs lightweight, disposable entity extraction rules and batching mechanisms that do not require persistent high-compute resources. These preliminary processing steps use minimal computational resources to prepare data for LLM verification, reducing the need for expensive, always-on hardware infrastructure.
3Device complexity
If the context window size is limited, then model size is reduced, but processing of long unstructured documents becomes difficult
Solution Approach 1:
The patent segments long unstructured documents into smaller batches or chunks that fit within the LLM's context window. By dividing documents into manageable segments and processing them sequentially through the two-phase approach, the system maintains versatility in handling long documents while adhering to the model's size and context constraints.
Solution Approach 2:
The patent performs preliminary entity extraction and batching actions before LLM processing. This preliminary action identifies and isolates relevant text segments that need LLM verification, ensuring that the LLM receives appropriately sized inputs that fit within its context window while still capturing essential information from long documents.
Data Source
AI summary
A system and method for classification. A method includes identifying candidate entities among text data by applying at least one entity identification rule to the text data. Inputs are constructed based on the identified candidate entities, where each input includes a first portion of text indicating a candidate entity and at least one second portion of text and where the at least one second portion of text of each input is adjacent to the first portion of text of the input. Multiple language models are applied to the inputs, where each language model is trained to identify a respective set of entities and where outputs of the language models include at least one portion of entity-indicating text for each input. Based on the outputs of the language models, at least one named entity in the text data is determined.