Intent Prediction Using Vector Clustering and Dimensionality Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in efficiently predicting the intent behind large volumes of customer communications, such as phone conversations, emails, and chat app communications, which hinders effective customer service improvement.
Innovation Solution
A method involving a customer utterance analysis system that processes a subset of utterances by encoding them into multi-dimensional vectors, reducing dimensionality, performing clustering, and training a neural network model to predict the intent of utterances based on labeled clusters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional methods are used to process large volumes of customer communications, then all communications can be analyzed, but the processing time and computational resources required become excessive
Solution Approach 1:
The patent segments the large volume of customer communications into smaller batches or subsets that can be processed independently and in parallel. This segmentation allows the system to analyze communications incrementally rather than all at once, reducing the time penalty associated with processing large datasets while maintaining comprehensive coverage of all inputs.
Solution Approach 2:
The patent performs preliminary processing steps such as encoding communications into vector representations and reducing dimensionality before the main intent prediction analysis. By preparing the data in advance through these preprocessing steps, the system reduces the computational burden during the actual prediction phase, thereby decreasing overall processing time while preserving accuracy.
2Productivity
If dimensionality reduction is applied to encoded vectors, then computational efficiency improves, but information loss may occur
Solution Approach 1:
The patent changes the parameter of vector dimensionality by applying dimensionality reduction techniques to the encoded vectors. This transformation reduces the number of features from a high-dimensional space to a lower-dimensional space, improving computational efficiency. The system carefully selects reduction techniques that preserve the most important information while discarding redundant or less significant dimensions, thereby minimizing information loss.
3Loss of information
If clustering is performed on reduced-dimensionality vectors, then patterns in customer communications can be identified, but the complexity of the analysis process increases
Solution Approach 1:
The patent applies clustering algorithms to the reduced-dimensionality vectors to identify patterns and group similar customer communications together. By working in the reduced dimensionality space rather than the original high-dimensional space, the clustering process becomes more computationally feasible while still effectively capturing the essential patterns and relationships in the data. This dimensional transformation enables pattern recognition without prohibitively increasing analysis complexity.
Data Source
AI summary
A method of operating a customer utterance analysis system includes obtaining a subset of utterances from among a first set of utterances. The method includes encoding, by a sentence encoder, the subset of utterances into multi-dimensional vectors. The method includes generating reduced-dimensionality vectors by reducing a dimensionality of the multi-dimensional vectors. Each vector of the reduced-dimensionality vectors corresponds to an utterance from among the subset of utterances. The method includes performing clustering on the reduced-dimensionality vectors. The method includes, based on the clustering performed on the reduced-dimensionality vectors, arranging the subset of utterances into clusters. The method includes obtaining labels for a least two clusters from among the clusters. The method includes generating training data based on the obtained labels. The method includes training a neural network model to predict an intent of an utterance based on the training data.