Transformer Model Matching Unstructured Text to Chat Conversations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional machine learning approaches for processing natural language in online conversations are suboptimal due to their unsupervised nature and reliance on single input queries, failing to effectively match user computers to relevant chat conversations in multi-message contexts, which is crucial for applications like merchant websites and search engines.
Innovation Solution
A computer-implemented method using a trained transformer-based machine learning model, specifically a supervised RoBERTa language model, processes unstructured text inputs to match user computers to relevant online chat conversations by generating prediction outputs with probability values, allowing for continuous re-training based on feedback and improving similarity assessment techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If unsupervised text retrieval approaches are used, then the system can process natural language text without requiring labeled training data, but the accuracy of matching users to relevant conversations deteriorates
Solution Approach 1:
The system implements feedback mechanisms where user interactions and conversation data are continuously used to retrain the transformer model, improving matching accuracy over time while maintaining initial ease of deployment
Solution Approach 2:
The system transitions from static unsupervised parameters to dynamic supervised parameters by continuously updating the transformer model with labeled data from user interactions, enabling progressive improvement in matching precision
2Device complexity
If conventional unsupervised approaches are used, then the system avoids complex training requirements, but it cannot continuously improve by updating training sets to address past errors
Solution Approach 1:
The system establishes continuous improvement through ongoing retraining cycles where the transformer model processes new conversation data and user feedback indefinitely, transitioning from one-time unsupervised setup to continuous supervised learning
Solution Approach 2:
User interactions and conversation outcomes provide feedback loops that enable the system to identify and correct matching errors, continuously improving reliability through iterative retraining on labeled data from past errors
3Productivity
If single input query classification is used, then the system simplifies processing by focusing on one query at a time, but it fails to capture the context and topic relationships in multi-message chat conversations
Solution Approach 1:
The system merges multiple individual chat messages into a unified contextual representation using the transformer model, allowing the system to process conversation context as an integrated whole rather than isolated queries, thereby improving contextual understanding while maintaining processing efficiency
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer joining an online chat service, based on unstructured text input, can be matched automatically under computer control to one of multiple different online chat conversations using a trained transformer-based machine learning model, training techniques, and similarity assessment techniques. Computer analysis in this manner improves the likelihood that the unstructured text input results in assigning the computer to a relevant chat conversation. Additionally, or alternatively, a dense passage retrieval machine learning model having a first encoder for resources and a second encoder for messages can automatically match relevant resources to computers or sessions based on analysis of a series of messages of an online chat conversation. In either approach, continuous re-training is supported based on feedback from a moderator computer and/or user computers.