Dialogue State Machine for Controlled LLM Function Calling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing dialogue systems face challenges in controlling conversational outputs, particularly when users deviate from predetermined designs, and large language models lack provider control, leading to unpredictable and inaccurate responses.
Innovation Solution
A dialogue management module overlays a large language model, using a deterministic state machine to guide conversational flow, integrating a RAG module for context retrieval and a dialogue state module to track history, enabling controlled and accurate responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a large language model is used to respond to unexpected inputs, then the system's adaptability improves, but the provider loses control over the conversational agent and hallucinations increase
Solution Approach 1:
The patent introduces an intermediary layer between the user input and the large language model, consisting of a state machine and dialogue management module. This intermediary processes inputs, manages conversation state, and controls what is passed to the LLM, thereby maintaining provider control while still allowing the system to handle unexpected inputs through the structured dialogue framework.
Solution Approach 2:
The system segments the dialogue processing into distinct components: a deterministic state machine for controlling conversation flow, a dialogue management module for state tracking, and the large language model for generating responses. This segmentation allows each component to perform its specific function, with the state machine ensuring reliability and the LLM providing adaptability.
2Adaptability or versatility
If a large language model is used to handle diverse tasks, then the system's versatility improves, but the output becomes unpredictable and inaccurate
Solution Approach 1:
The patent implements feedback mechanisms where the dialogue management module continuously monitors conversation state and adjusts the interaction with the large language model accordingly. The state machine provides feedback loops that verify outputs against expected patterns, allowing the system to maintain accuracy while handling diverse tasks through iterative refinement and state-based control.
3Reliability
If handcrafted intents and entities are used in intent detection, then the system's control improves, but the system fails when tasks fall outside predetermined design
Solution Approach 1:
The patent merges the deterministic control of handcrafted intent detection with the flexibility of large language models. The state machine maintains structured control over dialogue flow while the LLM component handles unexpected inputs and out-of-scope behaviors, creating a hybrid system that combines the strengths of both approaches.
Data Source
AI summary
A computer-implemented method of controlling an output from a dialogue system, the method comprising:receiving, by way of an input, first input data relating to speech or text provided by a user;selecting a first state from a plurality of states of a deterministic model, at least some states of the plurality of states being associated with a corresponding portion of a language model prompt including an instruction to call a corresponding function;responsive to the first state being associated with a corresponding portion, generating a first language model prompt comprising at least part of the corresponding portion associated with the selected first state;providing the first language model prompt as input to a language model to generate a first language model output;determining whether to execute a function based on the first language model output;responsive to determining to execute a first function based on the first language model output, executing the determined first function to generate a first function output;selecting a second state from the plurality of states based on the first function output;responsive to the second state being associated with a corresponding portion of a language model prompt, generating a second language model prompt comprising at least part of the corresponding portion associated with the selected second state;providing the second language model prompt as input to the language model to generate a second language model output;determining whether to provide an output to the user based on the second language model output; andresponsive to determining to provide an output to the user based on the second language model output, outputting, by way of an output, speech or text to the user.