Supervised Machine Learning for Automated Assistant Transcript Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automated assistants in financial institutions face challenges in accurately interpreting user queries in natural language, leading to inconsistent topic, subtopic, and intent classification, which affects the responsiveness and accuracy of financial account management operations.
Innovation Solution
The implementation of supervised machine learning models trained through automated workflows that analyze user interactions, identify topics, subtopics, and intents, and perform responsive actions, with a record verification process to address classification conflicts and model imbalances, ensuring accurate data representation and model deployment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If automated assistants use natural language processing to interpret user queries, then user interaction capability is improved, but classification accuracy and consistency deteriorate
Solution Approach 1:
The system performs preliminary actions by collecting and storing multiple transcript records of user queries with their corresponding topics, subtopics, and intents before actual classification occurs. This pre-collected training data enables the automated assistant to learn from diverse examples and improve classification accuracy while maintaining natural language interpretation capabilities.
Solution Approach 2:
The system implements feedback mechanisms by continuously monitoring classification performance and using transcript reviewer corrections to refine future classifications. The automated assistant learns from feedback loops where misclassified queries are reviewed, corrected, and used to improve the classification model, thereby maintaining both natural language versatility and classification precision.
2Reliability
If automated assistants process more user interactions to improve model accuracy, then classification reliability is improved, but data imbalance and conflicts increase
Solution Approach 1:
The system applies self-service principles by implementing automated conflict detection and resolution mechanisms. The automated assistant independently identifies classification conflicts in transcript records and resolves them through predefined rules and algorithms, reducing the need for manual intervention and managing data complexity autonomously while improving classification reliability.
Solution Approach 2:
The system changes parameters by dynamically adjusting data selection criteria and balancing techniques based on detected data imbalances. When certain topics or intents are underrepresented, the system modifies sampling parameters and weighting factors to achieve balanced training datasets, thereby improving reliability without proportionally increasing management complexity.
3Manufacturing precision
If transcript reviewers manually verify all user interactions, then data accuracy is improved, but processing time and operational complexity increase
Solution Approach 1:
The system applies partial action by having transcript reviewers verify only a subset of transcript records rather than all of them. The automated assistant handles routine classifications independently, while reviewers focus on edge cases, conflicts, or low-confidence predictions. This selective verification approach maintains data accuracy while significantly reducing the time and operational complexity of manual review.
4Measurement precision
If the automated assistant uses complex machine learning models to improve classification precision, then topic and intent identification accuracy is improved, but model training time and computational resources increase
Solution Approach 1:
The system performs preliminary actions by pre-processing and organizing training data before model training begins. Transcript records are collected, standardized, and balanced in advance, which reduces the computational burden during actual model training and deployment. This pre-preparation enables the use of complex machine learning models without proportionally increasing training time and resource requirements.
Data Source
AI summary
Disclosed are methods and systems for supervised machine learning for automated assistants. An example method includes: receiving an automated assistant transcript comprising a plurality of records, wherein each record of the plurality of records comprises a query, a classification of the query, an intent associated with the query, and a responsive action associated with the intent; receiving, via a graphical user interface (GUI), a user input indicating an approval of a new automated assistant transcript record; comparing the new automated assistant transcript record to one or more records of the plurality of records; and responsive to detecting a conflict of the new automated assistant transcript record with one or more records of the plurality of records, displaying, via the GUI, a notification of the conflict.