LLM Inference System Using Segmented Text Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large Language Models (LLMs) require significant data and processing power, making them expensive and inefficient for inferring topic, entity, and relationship information from unstructured text data, especially in large-scale deployments, and current troubleshooting methods are inadequate due to unclear data interpretation and intercorrelation.
Innovation Solution
A computing system and method that leverages LLMs by receiving unstructured data text, identifying topics and entities, filtering passages, and distilling inferred knowledge to efficiently infer relationships between entities and topics, using a processor and memory to execute LLMs on specific passages, reducing processing power requirements and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If Large Language Models (LLMs) are used to infer topic, entity, and relationship information from unstructured text data, then inference accuracy is improved, but computational cost and processing power requirements increase significantly
Solution Approach 1:
The patent segments the inference process into distinct stages: topic identification, entity recognition, and relationship extraction. Each stage processes specific aspects of the unstructured text data independently, allowing the system to leverage LLMs only when necessary while reducing overall computational burden compared to applying LLMs to the entire inference pipeline.
Solution Approach 2:
The patent introduces intermediate processing steps between raw text input and final inference output, including text preprocessing, feature extraction, and candidate generation. These intermediary components prepare data in a format that reduces the computational complexity required by LLMs, thereby maintaining inference accuracy while lowering computational costs.
2Productivity
If LLMs are deployed for large-scale inference tasks, then inference capability is improved, but deployment scalability is limited due to high processing power requirements
Solution Approach 1:
The system divides the large-scale inference task into manageable segments that can be processed in parallel or distributed across multiple computing resources. By segmenting the workload and applying LLMs only to critical sub-tasks rather than processing entire datasets with LLMs, the system achieves both high inference capability and improved deployment scalability.
Solution Approach 2:
The patent applies LLMs partially - only to specific passages, entities, or relationships that require advanced language understanding, rather than applying them uniformly to all data. This selective application maintains high inference capability for complex tasks while reducing overall computational requirements to enable large-scale deployment.
3Measurement precision
If LLMs process huge amounts of training data, then model performance is improved, but troubleshooting and bias resolution become complicated due to unpredictable data interpretation
Solution Approach 1:
The patent segments the training data processing into modular components with defined inputs and outputs. Each processing stage can be independently tested, validated, and debugged, making troubleshooting and bias resolution significantly easier compared to treating the entire LLM training process as a black box, while still achieving high model performance through cumulative processing.
Data Source
AI summary
The present computer system and method identify or infer a topic or an entity from unstructured data text, filter passages mentioning the topic and/or entity, and execute a Large Language Model (LLM) with the passages mentioning the topic for inferring at least one entity associated with the topic. The LLM may further infer a relationship between the topic and each of the at least one entity. The computer system and method further distil the topic, the at least one entity and the relationship therebetween into distilled inferred knowledge.