Chatbot Intent Refresh Using Conversation Timeline Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Chatbot performance deteriorates over time due to evolving user inquiries and new topics, requiring frequent updates to handle nuanced topic overlaps and improve response accuracy.
Innovation Solution
A system that analyzes conversation transcripts to identify changes in chatbot intents, ranks them for improvement, and updates the chatbot model based on these changes, using a virtual assistant analytic module to automate periodic refreshes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If chatbot intents are defined broadly to handle multiple topics, then the chatbot can respond to more user inquiries, but the precision of intent recognition deteriorates
Solution Approach 1:
The patent segments the chatbot's intent recognition system into multiple specialized intent models, each trained on specific topics or domains. Instead of using a single broad intent definition, the system divides intents into granular categories (e.g., separate intents for different product categories, services, or user goals). This segmentation allows the chatbot to maintain high precision within each intent category while collectively covering a broad range of topics through multiple intents.
2Measurement precision
If chatbot intents are defined narrowly to improve precision, then intent recognition accuracy improves, but the chatbot cannot handle diverse user inquiries
Solution Approach 1:
The patent implements a multi-functional intent recognition system where a single chatbot architecture can handle multiple diverse user inquiries through a collection of specialized intents. Each intent is narrowly defined for precision, but the aggregate of all intents provides universal coverage. The system uses a unified training framework that can adaptively select and combine multiple narrow intents to respond to various user goals, achieving both precision and versatility.
3Reliability
If the chatbot is updated frequently to handle new topics and nuances, then the chatbot performance improves, but the complexity of maintenance increases
Solution Approach 1:
The patent implements periodic updates to the chatbot's intent definitions and training data. Instead of continuous manual adjustments, the system schedules regular refresh cycles where intent models are retrained using accumulated conversation data. This periodic action maintains high performance by incorporating new topics and nuances at scheduled intervals, while reducing maintenance complexity by automating the update process and avoiding ad-hoc modifications.
Solution Approach 2:
The patent enables the chatbot system to automatically identify when updates are needed based on performance metrics and user interaction patterns. The system self-monitors its own performance, automatically triggers retraining when degradation is detected, and performs self-updates without requiring manual intervention for every change. This self-service capability maintains high reliability while minimizing maintenance complexity by eliminating the need for constant human oversight.
4Measurement precision
If experimental cycles are extended to learn the right balance of intent depth and breadth, then intent definition accuracy improves, but the time to deploy improves
Solution Approach 1:
The patent performs preliminary analysis of user conversation data to pre-identify potential intent definitions and their optimal granularity before full deployment. The system conducts preliminary experiments on subsets of data to determine the right balance of intent depth and breadth in advance, then applies these pre-determined intent definitions to the full chatbot deployment. This preliminary action reduces the need for extended experimental cycles after deployment, achieving accurate intent definitions faster.
Data Source
AI summary
Aspects of the present disclosure relate generally to chatbot performance and, more particularly, to periodically refreshing chatbots from timeline analytics of user online conversations to improve performance. In embodiments, a method includes: receiving, by a computing device, a plurality of conversation transcripts generated from a plurality of versions of a chatbot; determining, by the computing device, a plurality of changes of a plurality of attributes of intents between the plurality of the versions of the chatbot; identifying, by the computing device, at least one intent to update from the plurality of changes of the plurality of attributes of the intents to improve performance of the chatbot; and generating, by the computing device, another version of the chatbot that includes the at least one intent updated from the plurality of changes of the plurality of attributes of the intents to improve the performance of the chatbot.