LLM Chatbot Topic Filtering for Context Continuity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Chatbot applications based on large language models struggle to maintain focus on enterprise domain topics during free-flow natural language conversations, are prone to AI hallucinations, and are susceptible to prompt engineering jail break attempts, making it difficult to keep conversations directed and contextual.
Innovation Solution
A topical and contextual content filter for large language model chatbots that uses vector embeddings and a topic database to identify and maintain approved topics, resist prompt engineering, and integrate with existing systems to ensure consistent topical evaluation and context switching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a chatbot application allows free-flow natural language conversations, then user interaction flexibility is improved, but the chatbot loses focus on enterprise domain topics and becomes prone to AI hallucinations
Solution Approach 1:
The patent introduces a topical and contextual content filter as an intermediary component between the user input and the chatbot response generation. This filter includes a topical filter that identifies whether input relates to approved topics, and a contextual filter that determines if input continues the previous conversation context. Only inputs passing both filters are transmitted to the chatbot application, thereby maintaining conversation flexibility while ensuring topic focus and reducing AI hallucinations.
2Reliability
If the chatbot application uses a content filter to maintain topic focus, then reliability is improved, but system complexity increases
Solution Approach 1:
The content filter is segmented into distinct functional components: a topical filter that identifies approved topics using vector embeddings and a topic database, and a contextual filter that determines conversation continuity by comparing input with previous conversation history. This segmentation allows each component to perform its specific function independently, making the overall system more manageable and maintainable despite the increased complexity.
Solution Approach 2:
The system performs preliminary filtering of user input before it reaches the chatbot application. The topical and contextual filters evaluate inputs in advance, identifying and blocking potentially harmful or off-topic content before it can generate inappropriate responses. This preliminary action prevents AI hallucinations and maintains topic focus without requiring complex post-processing of chatbot outputs.
3Stability of the object's composition
If the chatbot application implements contextual filtering, then conversation continuity is maintained, but processing time increases
Solution Approach 1:
The contextual filter implements partial action by selectively processing only those inputs that require contextual evaluation. The filter first performs a quick topical assessment, and only inputs that pass the topical filter undergo the more time-consuming contextual analysis. This partial processing approach maintains contextual continuity for relevant conversations while minimizing processing time for clearly off-topic inputs.
Data Source
AI summary
Computer-implemented methods for a topical and contextual content filter for large language model chatbots. Aspects include receiving a user query of a user from a chatbot application. Aspects also include receiving a previous chatbot application conversation associated with the user. Aspects further include determining that the user query of the user from the chatbot application is not a continuation of the previous chatbot application conversation associated with the user. Aspects also include identifying a short list of possible topics using the user query and a topic database. Aspects include determining that the user query of the user from the chatbot application is not associated with the short list of possible topics. Aspects further include transmitting a notification to the chatbot application comprising a rejection of the user query.