Contextual Prompt Generation for Accurate API Actions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing natural language processing systems face inefficiencies in accurately determining appropriate actions from user inputs due to the need to process irrelevant information, leading to degraded accuracy and longer prompt lengths, especially when using language models for one-shot or few-shot learning.
Innovation Solution
A system is developed to generate a prompt for a language model that includes relevant API definitions, exemplars, device states, and contextual information, enabling efficient and accurate determination of actions by focusing on only the necessary information, thereby improving the language model's performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If the language model processes all available information including irrelevant data, then the model has access to comprehensive information, but the prompt length increases and processing efficiency decreases
Solution Approach 1:
The system segments the available information into relevant and irrelevant portions using a classifier. Only the relevant segments are included in the prompt presented to the language model, while irrelevant segments are filtered out. This segmentation resolves the contradiction by maintaining information completeness for relevant data while eliminating the burden of processing irrelevant information, thus improving processing efficiency without losing necessary information.
Solution Approach 2:
The system extracts only the relevant information from the complete set of available data using a trained classifier. By taking out and removing irrelevant information before presenting it to the language model, the system reduces prompt length and processing time while retaining all necessary information for accurate action determination.
2Measurement precision
If the prompt includes comprehensive information for accurate action determination, then the accuracy improves, but the prompt length increases leading to degraded model performance
Solution Approach 1:
The system segments information into relevant and irrelevant categories using a classifier, including only relevant segments in the prompt. This ensures the prompt contains sufficient information for accurate action determination while maintaining a manageable length that preserves language model performance.
Solution Approach 2:
The system changes the parameter of information selection by using a classifier to dynamically determine which information to include based on relevance. This parameter change allows the prompt to maintain optimal length while containing all necessary information for accurate action determination.
3Reliability
If the system processes all user input data, then no relevant information is missed, but the processing time and computational resources increase
Solution Approach 1:
The system performs preliminary action by using a classifier to pre-process and filter information before it reaches the language model. This preliminary filtering action ensures that only relevant information is processed further, maintaining reliability of action determination while significantly reducing processing time and computational resource requirements.
Solution Approach 2:
The system segments the processing pipeline into two stages: initial classification to identify relevant information, followed by language model processing only of that filtered information. This segmentation maintains reliability by ensuring no relevant information is missed while reducing overall processing time.
Data Source
AI summary
Techniques for generating a prompt for a language model to determine an action responsive to a user input, are described. In some embodiments, the system receives a user input, determines one or more application programming interfaces (APIs) configured to perform actions that are relevant to the user input and exemplars representing examples of using the APIs with respect to user inputs similar to the current user input. The system further determines device states of devices that are determined to be related to the user input and also determines other contextual information (e.g., weather information, time of day, geographic location, etc.). The system generates a prompt including the user input, the APIs, the exemplars, the device states, and the other contextual information. A language model processes the prompt to determine an action responsive to the user input and the system causes performance of the action.