Reactive AI Prompt Generation for Faster, More Reliable Conversations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing generative AI models suffer from slow performance in conversational settings due to the overhead of generating a full plan of action and difficulty in recovering from invalid or unexpected user input, leading to unacceptable user experiences in customer-facing scenarios like autonomous customer support chatbots, and often require large amounts of instructions that exceed input capacity.
Innovation Solution
An assistant service modularizes inputs to the generative AI model, employing a prompt engineering framework with system messages, interaction histories, and structured topics to weight important information more heavily, reducing prompt size and improving response accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a full plan of action is generated by the generative AI model, then the model can provide comprehensive responses, but the performance becomes slow due to overhead
Solution Approach 1:
The patent segments the prompt input into multiple structured components including system instructions, conversation history, and current user input. This segmentation allows the model to process information in organized chunks rather than attempting to generate a complete plan all at once, improving both speed and effectiveness of response generation.
Solution Approach 2:
The system performs preliminary actions by pre-processing and structuring the prompt before it reaches the generative AI model. The assistant service prepares the input data in advance with proper formatting and context organization, reducing the computational overhead during the actual response generation phase.
2Adaptability or versatility
If large amounts of instructions are provided to the generative AI model, then the model can handle complex tasks, but the input capacity is exceeded
Solution Approach 1:
The patent extracts only the essential and relevant information needed for the current task from the available context. The system selectively includes conversation history and instructions that are directly pertinent to the user's query, excluding redundant or less important information. This extraction approach maintains task handling capability while reducing the overall input volume to fit within model capacity limits.
3Ease of operation
If the generative AI model processes conversational input directly, then it can respond to user queries, but it struggles with invalid or unexpected input leading to poor user experience
Solution Approach 1:
The patent introduces an assistant service as an intermediary layer between the user input and the generative AI model. This intermediary service validates, structures, and prepares the input before it reaches the model, and also processes the model's output before presenting it to the user. This mediation improves the system's ability to handle invalid or unexpected input by adding layers of validation and error handling that enhance reliability while maintaining ease of operation.
Data Source
AI summary
Methods, apparatuses, and computer-program products are disclosed. The method may include generating a first system message indicative of a role for the generative artificial intelligence (AI) model; generating a query-response message pair that includes a query message that may include an action invocation and a response message that includes information responsive to the action invocation; obtaining one or more interaction messages; generating a second system message that includes an instruction for the generative AI model to generate an utterance and an indication of one or more actions available to the generative AI model; transmitting, to the generative AI model, a prompt that may include the first system message, the query-response message pair, the one or more interaction messages, and the second system message; and receiving, from the generative AI model and based on the prompt, an output of the generative AI model.