LLM Function Calling with Hierarchical Context Minimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Function calling for Large Language Models (LLMs) involves sending complete function lists and conversation histories, consuming the token budget and leading to errors or hallucinations, which existing techniques like prompt engineering do not effectively address.
Innovation Solution
Implementing a context interface system that minimizes token usage by using a hierarchical function group structure and private LLM functions, reducing the amount of data exchanged with the LLM while maintaining functionality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If complete function lists and conversation histories are sent to LLM for function calling, then the LLM can classify intent and invoke functions, but the token budget is consumed leading to errors or hallucinations
Solution Approach 1:
The patent segments the complete function list into function groups with hierarchical structures. Instead of sending all functions at once, the system divides functions into categories (e.g., weather functions, travel functions) and sends only relevant groups to the LLM based on conversation context, thereby reducing token consumption while maintaining full function calling capability.
Solution Approach 2:
The patent implements dynamic function group selection where the system adapts which function groups to send to the LLM based on conversation history and detected intent. The function groups are dynamically added or removed from the context window, allowing the system to optimize token usage while preserving adaptability for different conversation scenarios.
2Loss of information
If large function lists are combined with lengthy conversation history, then the LLM has comprehensive context, but the token limit is exceeded causing errors
Solution Approach 1:
The patent extracts only the essential conversation history needed for function calling decisions, removing redundant or irrelevant dialogue turns. The system identifies and retains only the critical context (e.g., user intent, previous function calls, relevant responses) while discarding extraneous information, thus maintaining context completeness within token limits and improving reliability.
Solution Approach 2:
The patent applies partial action by sending only a subset of conversation history that is sufficient for function calling rather than the complete history. The system determines the optimal truncation point where enough context is provided for accurate function selection without exceeding token limits, balancing context completeness with error prevention.
3Measurement precision
If the LLM is provided with extensive context data, then it can make informed function calls, but it may hallucinate due to token budget constraints
Solution Approach 1:
The patent performs preliminary processing of function lists into structured groups with clear descriptions and categories before presenting them to the LLM. This pre-organization of information enhances the LLM's ability to accurately classify intent and select functions, improving measurement precision while reducing the likelihood of hallucination through better information structuring.
Data Source
AI summary
Provided herein are techniques to facilitate minimizing context for LLM function calling operations. In one example, a method may include obtaining, by an interface system, a user conversation input; providing, to an LLM, for each of one or more interactions between the interface system and the LLM: a description of the interface system; the user conversation input; a list of function groups that describes types of functions that the interface system is capable of performing; and a list of private functions that are callable by the LLM, based on the user conversation input, to receive function context from the interface system or to cause function operations to be performed by the interface system; and providing one or more user conversation outputs based on at least one of the one or more interactions by the interface system with the LLM.