Context Variable Mechanism for Passing Complex Data in LLM Processes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large Language Models (LLMs) face challenges in processing and outputting meaningful responses to prompts containing 'non-language text' or complex data, which can result in erroneous outputs or exceed prompt length limits.
Innovation Solution
The method involves constructing a prompt based on user input, using an orchestrator to select AI plugins and form context variables to represent complex data, and modifying the prompt to include references to these context variables without including the complex data itself, allowing for efficient passing of complex data between AI plugins and generative models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If complex data is included directly in the prompt input to LLMs, then the LLM can process and output meaningful responses, but the prompt length may exceed token limits and erroneous outputs may occur
Solution Approach 1:
The patent segments the data into two distinct parts: a compressed representation (context variable name) that is passed to the LLM, and the full complex data object that is stored separately and accessed via the context variable. This segmentation allows the prompt to remain within token limits while the LLM can still access meaningful information about the data through the compressed reference.
Solution Approach 2:
The patent extracts only the essential identifier (context variable name) from the complex data object and passes this extracted reference to the LLM, while leaving the full complex data object outside the prompt. This extraction approach reduces prompt length significantly while maintaining the ability to access meaningful data through the reference.
2Adaptability or versatility
If complex data is passed between AI plugins sequentially, then data can be shared between plugins, but processing speed decreases due to sequential operations
Solution Approach 1:
The patent creates a universal context variable mechanism that serves multiple functions simultaneously: it acts as a data reference for LLMs, enables data sharing between AI plugins, and provides a standardized interface for accessing complex data. This multi-functionality allows different components to access shared data without requiring sequential passing, thereby improving processing speed while maintaining versatility.
3Length of moving object
If context variables are used to reference complex data, then prompt length is reduced and token limits are avoided, but the system complexity increases due to additional context management
Solution Approach 1:
The patent introduces context variables as intermediary references that mediate between the complex data objects and the LLM. These context variables serve as simple, manageable placeholders that the LLM can work with without needing to handle the full complexity of the underlying data structures, thereby reducing prompt length while the intermediary layer manages the complexity of data access.
Data Source
AI summary
Examples are disclosed that relate to passing complex data objects as context variables between iterative prompts to AI models. One example provides a method, comprising receiving an input, constructing a prompt based upon the input, and inputting the prompt into an orchestrator. The method further comprises, based on the prompt, forming a context variable to reference complex data. The method further comprises providing modified input to a first AI plugin at a first orchestration stage, receiving the complex data from the first AI plugin, and storing the complex data as the context variable. The method further comprises modifying the prompt to include a reference to the context variable without modifying the prompt to include the complex data, thereby forming a modified prompt. The method further comprises providing the modified prompt to a generative model, receiving generated text from the generative model, and outputting the generated text with the complex data.