ML Outlier Classification via Similarity Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning-based classification techniques face challenges in reliably classifying data samples that are outliers or anomalies, leading to reduced performance and increased human intervention, especially when data samples are unevenly distributed in feature space.
Innovation Solution
A method and system that utilize a validating ML-based model to calculate a similarity metric value between incoming data samples and previously classified samples, validating the assignment of classes based on this metric, and optionally performing supplementary training to improve the classification model's performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a deterministic classification model is used to check for exact matches, then misclassifications are avoided, but the approach becomes ineffective due to numerous user request variations
Solution Approach 1:
The patent creates synthetic training samples by copying and transforming existing labeled data samples. It generates paraphrased versions of user requests with the same intended meaning but different formulations, thereby expanding the training dataset's coverage of request variations without requiring manual labeling of new examples
Solution Approach 2:
The patent performs preliminary data preparation by pre-generating synthetic training samples before the model training phase. This includes pre-creating paraphrased versions of requests and pre-labeling them, so that when the model encounters similar variations during inference, it can classify them accurately without requiring exact matches
2Productivity
If the confidence metric threshold is lowered to accommodate outliers, then more requests are classified automatically, but the number of errors increases
Solution Approach 1:
The patent applies partial action by selectively lowering the confidence threshold only for specific cases where synthetic training samples provide strong evidence of similarity. Instead of uniformly lowering thresholds across all requests, the system maintains high thresholds for standard cases while being more permissive for outliers that match synthetic samples, thereby balancing productivity and reliability
3Reliability
If the training dataset is increased to improve model performance, then classification reliability improves, but data collection time increases
Solution Approach 1:
The patent uses copying to generate synthetic training samples from existing labeled data. By automatically creating paraphrased versions and transformations of existing samples, it expands the training dataset size significantly without the time-consuming process of manual data collection and labeling, thus improving model performance while minimizing time loss
Solution Approach 2:
The system performs self-service by automatically generating its own training data through synthetic sample creation. The model training pipeline includes automated synthesis of training examples with proper labeling, eliminating the need for external manual data collection efforts and reducing the time investment required to improve model performance
Data Source
AI summary
A method and a system for improving classification of data samples, which may be considered as class outliers, are claimed. The method includes inferring a pretrained classifying ML-based model on the incoming data sample, to assign a particular class of a plurality of classes thereto; calculating a similarity metric value representing a degree of similarity between the incoming data sample and one or more previously classified data samples of the particular class; and validating assignment of the particular class to the incoming data sample, based on the calculated similarity metric value.