LLM Tagging Pipeline for Accurate Financial Document Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI models, such as those from OpenAI, Meta, and Anthropic, are ineffective in accurately analyzing and classifying financial documents like SEC filings, often failing to respond, providing incorrect responses, or generating hallucinations.
Innovation Solution
A method involving fine-tuning pre-trained large language models (LLMs) with a data extraction pipeline that systematically generates training messages for tag identification, text identification, and tag learning, using XBRL tags, and adjusting model weights to enhance document classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If pre-trained LLMs are used for financial document analysis, then general language understanding is provided, but accuracy and reliability deteriorate due to hallucinations and incorrect responses
Solution Approach 1:
The patent applies preliminary action by pre-processing financial documents to extract structured data, tags, and context information before training the LLM. This includes extracting XBRL tags, identifying financial concepts, and preparing labeled datasets that capture the nuanced relationships in financial terminology, thereby improving reliability without requiring complex training procedures
Solution Approach 2:
The patent introduces an intermediary layer of domain-specific embeddings and financial concept representations that bridge the gap between general LLM knowledge and specialized financial data. This intermediary structure allows the model to accurately classify financial documents by translating them into a standardized financial concept space, resolving the contradiction between general model capability and specialized accuracy
2Measurement precision
If LLMs are trained on specialized financial data, then classification accuracy improves, but training complexity and data processing requirements increase
Solution Approach 1:
The patent segments the complex training process into distinct modular stages: document parsing, XBRL tag extraction, financial concept identification, context window generation, and model training. Each stage processes specific aspects of the data independently, making the overall system more manageable and maintainable while achieving high precision in tag identification through specialized processing at each segment
Solution Approach 2:
The patent utilizes parameter changes by dynamically adjusting context window sizes and embedding dimensions based on the complexity of financial documents and the specific classification task requirements. This allows the system to maintain high measurement precision for tag identification while optimizing computational resources and reducing unnecessary complexity in the data extraction pipeline
Data Source
AI summary
The present disclosure includes gathering a tagged document of a type, collecting a repository of tags pertaining to the predetermined type of document, providing the tagged document and the repository of tags to train a first pre-trained LLM, identifying a first tag in the gathered document, pairing one text with the first tag, identifying one value associated with the paired first tag and the text, formatting the paired first tag and the text and the associated value to form a training message to train a second pre-trained LLM, providing an unseen document of the type to the first trained LLM, generating, via executing the first trained LLM, a second tag from the unseen document, providing the second tag and the unseen document to the second trained LLM, and identifying, via executing the second trained LLM, an unseen text paired with the second tag and an associated value.