Arbitrary-Length String Classification with XBRL-Aware LLMs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AI models, such as those from OpenAI, Meta, and Anthropic, are ineffective in accurately analyzing and classifying financial documents like SEC filings, often failing to respond, providing incorrect responses, or generating hallucinations.
Innovation Solution
A method involving fine-tuning pre-trained large language models (LLMs) using a data extraction pipeline to identify and pair tags with text in financial documents, adjusting model weights, and employing reinforcement learning and human feedback to improve accuracy, with a focus on XBRL tags for financial reports.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current pre-trained LLMs are used for financial document analysis, then general language understanding capability is maintained, but accuracy in classifying financial data and tagging XBRL elements deteriorates
Solution Approach 1:
The patent applies preliminary action by pre-processing financial documents into structured training datasets with XBRL tags and contextual information before model training. This preparatory step organizes raw financial documents into a format that directly teaches the model domain-specific patterns, improving classification accuracy before the model encounters real-world financial documents.
Solution Approach 2:
The patent changes key training parameters including chunk size (adjusted from initial to optimized values), context window size (set to 384 tokens), and training data composition (incorporating tagged documents with XBRL elements). These parameter adjustments transform the model's behavior to better suit financial document analysis while maintaining general language capabilities.
2Productivity
If document text is divided into small chunks for training, then model training efficiency is improved, but context understanding capability deteriorates
Solution Approach 1:
The patent implements nesting by embedding multiple levels of context within training examples: individual text chunks are nested within larger document contexts, which are themselves nested within structured training datasets containing XBRL tags and metadata. This nested structure allows the model to learn from both fine-grained local patterns and coarse-grained global context simultaneously.
Solution Approach 2:
The patent adds dimensional depth to training data by incorporating contextual information from surrounding text, XBRL tags, and document metadata. Instead of training on flat text chunks alone, the model receives multi-dimensional input that includes positional context, semantic context from adjacent passages, and structured financial data elements, enabling efficient learning without context loss.
3Measurement precision
If model size is increased to improve classification accuracy, then precision in financial data tagging is improved, but computational resources and training time increase
Solution Approach 1:
The patent applies local quality by fine-tuning only the necessary portions of the pre-trained model architecture for financial document classification while keeping other parameters frozen. This selective training approach concentrates computational resources on the specific layers and parameters that directly impact XBRL tagging accuracy, achieving high precision without requiring a complete increase in model size.
Solution Approach 2:
The patent uses partial action by training on a curated subset of financial documents with representative XBRL tags and contexts rather than exhaustively training on all possible financial document types. This partial training approach achieves sufficient precision for the target application while significantly reducing computational resource requirements compared to comprehensive training.
Data Source
AI summary
The present disclosure includes gathering a tagged document of a type, collecting a repository of tags pertaining to the predetermined type of document, providing the tagged document and the repository of tags to train a first pre-trained LLM, identifying a first tag in the gathered document, pairing one text with the first tag, identifying one value associated with the paired first tag and the text, formatting the paired first tag and the text and the associated value to form a training message to train a second pre-trained LLM, providing an unseen document of the type to the first trained LLM, generating, via executing the first trained LLM, a second tag from the unseen document, providing the second tag and the unseen document to the second trained LLM, and identifying, via executing the second trained LLM, an unseen text paired with the second tag and an associated value.