Chatbot Reliability Assessment for Semi-Automatic Messaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current chatbot systems often provide inappropriate answers to questions they cannot mechanically respond to, leading to inefficiencies in customer communication.
Innovation Solution
A semi-automatic communication system that uses a chatbot and a consultant device, where the chatbot provides an initial answer and the reliability is determined; if high, the chatbot's answer is used, and if low, the consultant device provides a manual answer, with both interactions being stored for machine learning to improve future chatbot responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a chatbot automatically responds to all user messages, then productivity is improved, but reliability deteriorates when the chatbot cannot mechanically respond to certain questions
Solution Approach 1:
The patent introduces a reliability determination unit as an intermediary between the chatbot and the user. This unit evaluates the reliability of chatbot-generated answers and acts as a gatekeeper, deciding whether to pass the automatic answer to the user or transfer the query to a consultant. This resolves the contradiction by filtering out low-reliability automatic responses that would otherwise harm answer accuracy.
Solution Approach 2:
The system dynamically adjusts the response mechanism based on the reliability of the chatbot's answer. When reliability is high, the chatbot responds automatically; when reliability is low, the system switches to human consultant response. This dynamic adaptation allows the system to maintain high productivity for straightforward queries while ensuring reliability for complex or uncertain queries.
2Productivity
If a chatbot provides automatic answers to all messages, then efficiency is improved, but manufacturing precision of answers deteriorates
Solution Approach 1:
The patent implements a feedback mechanism where user responses and consultant answers are fed back into the system. The reliability determination unit learns from these feedbacks to improve its reliability assessment, and the chatbot continuously trains on consultant-provided correct answers, enhancing answer quality over time while maintaining efficient automatic responses.
Solution Approach 2:
The patent replaces the purely mechanical chatbot response system with a hybrid system that incorporates intelligent reliability assessment. Instead of mechanically responding to all queries, the system uses the reliability determination unit to intelligently filter and select which queries should receive automatic answers, thereby improving answer quality without sacrificing overall communication efficiency.
3Reliability
If a consultant device handles all user messages manually, then reliability is improved, but productivity deteriorates
Solution Approach 1:
The patent segments the user message handling process into two distinct pathways: automatic handling by the chatbot for high-reliability queries and manual handling by consultants for low-reliability queries. This segmentation allows the system to maintain high reliability through consultant involvement while preserving productivity through automatic handling of suitable queries.
Solution Approach 2:
The reliability determination unit performs preliminary assessment of each user message before it reaches the consultant. By pre-evaluating whether a message is suitable for automatic handling, the system filters out messages that can be efficiently handled by the chatbot, allowing consultants to focus only on messages requiring human judgment, thereby maintaining both reliability and productivity.
4Manufacturing precision
If a semi-automatic system with reliability determination is implemented, then answer quality is improved, but device complexity increases
Solution Approach 1:
The reliability determination unit serves multiple functions: it evaluates chatbot answer reliability, decides whether to transfer queries to consultants, and collects data for continuous improvement. This multi-functionality reduces the need for separate specialized components, thereby limiting the increase in system complexity while achieving improved answer quality.
Data Source
AI summary
A method for providing semi-automatic communication using a chatbot and a consultant device includes receiving, from a messenger server, a message input to an instant messaging application of a user device; determining a reliability for an automatic answer message made by the chatbot to answer the message; and transmitting the automatic answer message to the user device through the messenger server if the reliability is determined higher than a predetermined level, or enabling the consultant device to transmit a manual answer to the message if the reliability is determined lower than the predetermined level. The reliability is calculated based on a result of analysis for the message.