Language Model Function Invocation via Segmented Prompting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing language models are resource-intensive and prone to errors due to hallucinations, especially when interacting with a large number of functions.
Innovation Solution
A technique that sends targeted prompts to a language model, specifying a query and selector information to identify suitable functions, then provides more detailed information for the selected functions to generate proper API messages, thereby reducing resource usage and error rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If complete function definitions for all functions are provided to the language model, then the language model has comprehensive information to select functions, but memory resources and processor resources are consumed excessively
Solution Approach 1:
The patent segments the function library into multiple groups and provides only selected function definitions to the language model based on the query, rather than providing all function definitions at once. This segmentation approach reduces the token count in prompts while maintaining sufficient information for accurate function selection.
Solution Approach 2:
The patent applies partial action by providing only the necessary function definitions (a subset) rather than all function definitions. The system determines which function definitions are needed based on the query and selects appropriate functions from the provided subset, avoiding the excessive resource consumption of processing all possible function definitions.
2Loss of information
If all function definitions are included in prompts, then the language model receives complete information, but the number of tokens increases leading to higher resource consumption
Solution Approach 1:
The patent extracts only the necessary function definitions from the complete function library and includes them in the prompts. The system determines which function definitions are relevant to the current query and extracts only those, reducing the total token count while maintaining information completeness for the selected functions.
Solution Approach 2:
The function library is segmented into multiple groups, and the patent provides only the relevant segment (function definitions) to the language model based on the query. This segmentation reduces the quantity of tokens in each prompt while ensuring that the language model receives complete information about the functions it needs to select from.
3Stability of the object's composition
If function-related information accumulates in context store during sessions, then historical context is preserved, but prompt length increases over time
Solution Approach 1:
The patent implements a mechanism that discards (removes) function-related information from the context store after a certain condition is met (e.g., after a set number of prompts or when the information is no longer needed). This prevents prompt length from increasing indefinitely while preserving necessary historical context for the duration of the session.
Solution Approach 2:
The system performs periodic pruning of the context store, removing function-related information at regular intervals or when specific triggering conditions are met. This periodic action maintains context persistence for the active session while preventing unbounded growth of prompt length.
Data Source
AI summary
A technique sends a first prompt to a language model that specifies selector information. The selector information provides a summary of a group of functions that are capable of being invoked. The language model responds by choosing one or more functions from the group of functions. The technique then sends a second prompt to the language model that specifies more detailed information regarding just the function(s) that have been identified by the language model. The language model responds by providing invocation information for each of the functions, such as properly formatted API messages. The technique then invokes the function(s) based on the invocation information. The technique reduces the size of each prompt sent to the language model, which makes efficient use of resources and improves the quality of the language model's output results.