Quantile-Based Thresholds for Predictive Model Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing predictive models in contact centers struggle to consistently classify customers as members of a target audience due to biased probability scores and drift over time, making it challenging for end users to set appropriate action thresholds for triggering desired response actions.
Innovation Solution
A computer-implemented method that records previous probability scores, groups and sorts them to derive reference quantiles, and uses these quantiles to determine action thresholds. This method compares new probability scores to the reference quantiles to classify customers as part of a target audience, ensuring consistent and effective targeting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional action thresholds are used for predictive model outputs, then the system can trigger response actions, but the classification of target audience becomes inconsistent and unreliable due to biased probability scores and drift over time
Solution Approach 1:
The patent transforms the probability score parameter into a quantile-based ranking system. Instead of using raw probability scores that drift over time, the system converts them into quantiles that represent relative positioning within the distribution, making the measurement invariant to model drift and bias
Solution Approach 2:
The patent introduces quantiles as an intermediary between the predictive model output and the action threshold decision. This intermediary layer (quantile transformation) decouples the model's probability estimates from the final classification decision, allowing consistent target audience identification even when model accuracy varies
2Adaptability or versatility
If fixed action thresholds are set for predictive models, then the system can operate with simple decision rules, but it fails to adapt to changing data distributions and model drift over time
Solution Approach 1:
The patent replaces static fixed thresholds with dynamic quantile-based thresholds that automatically adapt to changing data distributions. The system continuously recalculates quantiles from historical probability scores, allowing the thresholds to evolve with the model and data while maintaining relative performance standards
Solution Approach 2:
The patent performs preliminary transformation of probability scores into quantiles before applying thresholds. This pre-processing step creates a standardized reference frame that simplifies subsequent threshold application and enables consistent adaptability across different models and time periods
3Reliability
If quantile-based classification is implemented, then consistent target audience identification is achieved, but additional processing steps are required to derive and apply quantiles
Solution Approach 1:
The system performs self-calibration by automatically deriving quantiles from its own historical probability score distributions. This self-service approach eliminates the need for external manual threshold calibration and enables the system to autonomously adapt to its performance characteristics
Solution Approach 2:
The system implements feedback loops where historical classification results and probability scores are continuously fed back into the quantile derivation process. This feedback mechanism allows the system to learn from past performance and continuously refine its quantile-based classification accuracy
Data Source
AI summary
A method for classifying customers via operation of a probability score of a predictive model that expresses a probability of a customer achieving a predicted outcome. The method includes: recording previous probability scores and grouping within a dataset; sorting the dataset by an increasing value; deriving a set of reference quantiles based on the sorted previous probability scores of the dataset; determining action thresholds defining a range of target quantiles within the set of reference quantiles; receiving a probability score from the predictive model for a first customer in a current interaction; comparing the probability score of the first customer to the probability scores in the set of reference quantiles to determine a matching quantile; and determining if the matching quantile is within the range of target quantiles and, based thereon, selectively classifying the first customer in the target audience.