LLM Tool Documentation for Zero-Shot Usage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models struggle to learn effectively from demonstrations, which can lead to biased usage and are difficult to acquire, especially for complex tasks, making it challenging to determine how many and which demonstrations to provide.
Innovation Solution
Instead of relying on demonstrations, large language models learn from tool documentation, which provides neutral instructions on tool functionalities, allowing them to determine the tasks each tool is operable to perform and generate plans to comply with user requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If demonstrations are provided to teach large language models tool usage, then the models can learn tool usage, but it becomes hard to acquire demonstrations and can result in biased usage
Solution Approach 1:
The patent introduces tool documentation as an intermediary between the tool itself and the large language model. Instead of directly providing demonstrations (which are hard to acquire and may be biased), the system uses documentation as a neutral mediator that describes tool functionalities. This intermediary approach resolves the contradiction by making tool usage teaching more reliable without the acquisition difficulties and biases associated with demonstrations.
2Adaptability or versatility
If demonstrations are used to teach complex tasks, then the models can learn the tasks, but the selection search grows combinatorially and becomes intractable
Solution Approach 1:
The patent extracts the essential information needed for tool usage from complex demonstrations and consolidates it into structured tool documentation. By taking out only the necessary functional descriptions from elaborate demonstrations, the system maintains the ability to teach complex tasks while eliminating the combinatorial explosion of demonstration selection. The documentation format provides a standardized, manageable representation that scales better than demonstration-based approaches.
Data Source
AI summary
Using a large language model to comply with a user request. The large language model receives tool documentation for each of one or more tools, and analyzes the tool documentation for each of the one or more tools to determine, for each tool, one or more tasks that the tool is operable to perform. Upon receiving a request from a user, the large language model generates a plan for complying with the request by using one or more of the tools, the plan including performance of one or more of the tasks.