Fine-Tuned Language Models for Structured Data Context Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models (LLMs) struggle to integrate structured data effectively, leading to errors and inefficient use of computing resources due to the complex nature of structured data exceeding token limits and causing errors even within the maximum input context.
Innovation Solution
Fine-tuning pre-trained language models to determine relevant portions of structured data, such as tables, using an on-policy learning framework with rewards and penalties to optimize the selection of correct data, reducing the input context provided to LLMs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If structured data is provided as input context to LLM, then the model can access comprehensive information, but the input context exceeds token limits and causes errors
Solution Approach 1:
The patent extracts only the relevant portions of structured data needed to answer the query, removing unnecessary information. The fine-tuned language model identifies and extracts relevant rows and columns from tables, extracting only the essential data subset that fits within token limits while maintaining answer accuracy.
Solution Approach 2:
The patent segments structured data into relevant and irrelevant portions based on query requirements. The fine-tuned model divides the structured data into meaningful segments (relevant rows, relevant columns) and processes only those segments, rather than treating the entire structured data as a single unit.
2Reliability
If structured data is provided to LLM, then comprehensive information is available, but the model provides errors due to complex nature of structured data
Solution Approach 1:
The patent introduces a fine-tuned language model as an intermediary between the query and the structured data. This intermediary model is specifically trained to understand query-intent relationships and selectively retrieve relevant data, acting as a mediator that simplifies the complex interaction between natural language queries and structured data formats.
Solution Approach 2:
The patent performs preliminary processing of structured data by pre-identifying relevant rows and columns before the main LLM processes the information. The fine-tuned model performs this preliminary filtering action upfront, reducing the complexity of data that the main LLM needs to handle subsequently.
3Productivity
If fine-tuned language model reduces structured data, then computing resources are used efficiently, but the model must accurately select relevant portions
Solution Approach 1:
The patent implements feedback mechanisms during fine-tuning where the model learns from rewards and penalties based on selection accuracy. The fine-tuned language model receives feedback signals (rewards for correct selections, penalties for incorrect ones) that guide it to improve its selection precision while maintaining efficient processing of structured data.
Data Source
AI summary
Methods and systems are provided for using a fine-tuned language model to reduce representations of structured data. In embodiments described herein, training data is accessed that includes structured data, a set of queries for the structured data, and each portion of the structured data that is relevant to each query of the set of queries. A language model is fine-tuned to maximize a cumulative reward based on the training data. The cumulative reward includes a reward for determining a correct portion of the structured data, a first penalty for failing to determine the correct portion of the structured data, and a second penalty for determining an incorrect portion of the structured data. The fine-tuned language model is then output.