LLM Output Validation Using Weighted Text Relevancy Rules
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large Language Models (LLMs) suffer from computational constraints, hallucinations, inaccuracies, and limited knowledge, which adversely impact their operability and applicability across various industries.
Innovation Solution
A system and method for validating LLMs by extracting strings from documents, parsing tokens, applying part-of-speech labels, building weighted categorization rules, and computing relevancy scores to confirm the presence of information in documents, thereby enhancing the accuracy and reliability of LLM outputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If LLMs are used for content generation and information extraction, then productivity is improved, but accuracy and reliability deteriorate due to hallucinations and inaccuracies
Solution Approach 1:
The patent implements a validation feedback mechanism where the LLM's extracted strings are verified against the original documents using NLP techniques. The system parses tokens, applies part-of-speech labels, and computes relevancy scores to confirm whether extracted information actually exists in the source documents. This feedback loop detects and corrects hallucinations, thereby improving reliability while maintaining productivity.
Solution Approach 2:
The patent introduces an intermediary validation system between the LLM and the final output. This intermediary layer uses traditional NLP methods (tokenization, lemmatization, part-of-speech tagging, and categorization) to verify the LLM's extractions. The intermediary acts as a mediator that filters out hallucinated content while preserving valid extractions, thus improving accuracy without sacrificing the LLM's productive capabilities.
2Adaptability or versatility
If LLMs process complex language tasks, then adaptability is improved, but measurement precision deteriorates due to inability to verify extracted information
Solution Approach 1:
The patent segments the information extraction process into two distinct parts: the LLM performs flexible adaptation to different language tasks, while the NLP validation system handles precise verification. By segmenting the workflow, the system maintains the LLM's adaptability across various tasks while assigning measurement precision to the traditional NLP verification component that checks token matches and relevancy scores.
Solution Approach 2:
The validation system serves as an intermediary that bridges the gap between the LLM's flexible but imprecise extractions and the need for precise verification. It uses deterministic NLP techniques (exact token matching, lemmatization comparison, part-of-speech validation) to measure whether the LLM's adapted extractions actually exist in the source documents, thereby providing measurement precision for diverse language tasks.
3Reliability
If comprehensive validation is performed on LLM outputs, then reliability is improved, but device complexity increases due to additional processing steps
Solution Approach 1:
The validation system is designed to be self-contained and self-sufficient. It uses the LLM's own extracted strings as input and autonomously verifies them against the source documents using standard NLP pipelines. The system self-manages tokenization, lemmatization, part-of-speech tagging, and relevancy scoring without requiring external complex infrastructure, thereby improving reliability while controlling complexity through self-service architecture.
Solution Approach 2:
The validation system employs universal NLP components (tokenizers, lemmatizers, part-of-speech taggers) that can handle various types of LLM extractions across different domains and tasks. These multi-functional components perform multiple validation functions (exact matching, fuzzy matching, contextual verification) using the same underlying infrastructure, thereby achieving comprehensive validation without proportionally increasing system complexity.
Data Source
AI summary
A data processing system and method for validating a language model includes executing the language model to extract a plurality of strings from a set of documents and for each string, parsing the string to generate a plurality of tokens, lemmatizing the plurality of tokens, applying a part-of-speech label to the plurality of tokens, executing a filtering operation to obtain a plurality of filtered tokens, automatically building a weighted categorization rule based on the plurality of filtered tokens, applying the weighted categorization rule to the set of documents to identify one or more text spans from the set of documents, computing a relevancy score for each of the one or more text spans extracted from the set of documents, and selecting the one or more text spans extracted from the set of documents having the relevancy score greater than a predetermined threshold.