Chatbot Intent Training via Feedback Correlation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Chatbot systems often generate unhelpful responses due to incorrect intent selection or inability to select established intents with high confidence, leading to unsatisfactory user interactions, and fail to utilize valuable feedback from users for improved training.

Innovation Solution

The system identifies and associates new utterances with existing intents, using cognitive analysis and natural language processing to improve intent classification by scoring unsatisfactory interactions and incorporating successful subsequent utterances into training data, thereby enhancing chatbot performance over time.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If the chatbot uses a generic catch-all intent for unrecognized utterances, then the system can respond to any user input, but the response quality decreases and valuable training feedback is lost

Engineering Contradiction:

Improveability to respond to any user inputVSAvoidresponse quality and intent classification accuracy

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The system captures user feedback from interactions with the catch-all intent, including subsequent utterances that successfully resolve the user's intent. This feedback is stored and used to continuously improve the intent classification model, transforming the catch-all intent from a dead end into a learning opportunity that enhances overall system accuracy over time

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system proactively collects and processes training data from catch-all intent interactions before they would normally be discarded. By preparing and integrating this feedback data into the training set in advance, the system ensures that valuable learning opportunities are captured and utilized for improving intent classification, rather than waiting for manual intervention

Inventive Principle:

Principle #10Preliminary action

2Manufacturing precision

If the chatbot requires multiple utterances to correctly identify user intent, then the system can achieve more accurate responses, but the interaction time and user effort increase

Engineering Contradiction:

Improveintent classification accuracyVSAvoidinteraction time and user effort

Core Design Contradiction:

Manufacturing precisionVSLoss of time

Solution Approach 1:

The system automatically processes and integrates user feedback from multi-utterance interactions into the training data without requiring manual intervention. This self-service approach allows the system to continuously learn from real-world interactions, improving intent classification accuracy while maintaining efficient user experiences, as the learning happens autonomously in the background

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS11184298B2Methods and systems for improving chatbot intent training by correlating user feedback provided subsequent to a failed response to an initial user intent

Publication Date: 2021.11.23 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

Embodiments for managing chatbots are provided. Chatbot interaction information is received. At least one chatbot interaction within the received chatbot interaction information is identified. The at least one identified chatbot interaction includes an initial utterance provided by a user, an unsuccessful response generated responsive to the initial utterance, a subsequent utterance provided by the user, and a successful response generated responsive to the subsequent utterance. The successful response is associated with a chatbot intent. A confidence score for associating the initial utterance with utilization of the chatbot intent by a chatbot is calculated.