Proxy Model for Data Segmentation Drift Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In production environments, data evolution over time leads to existing data segmentation becoming less accurate, requiring frequent updates from subject matter experts (SMEs), which is costly and inefficient, especially in systems using supervised machine learning for operational data from network devices.
Innovation Solution
A method that uses a proxy model to provide low-latency, high-throughput labeling of new data points and adjusts segmentation parameters based on comparisons between initial and subsequent segmentations, reducing the need for SME input by detecting when existing labels are insufficient, and applying annotations via inference to maintain domain relevance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data segmentation is frequently updated to maintain accuracy with evolving data, then classification accuracy is improved, but operational expenses and computational burden increase
Solution Approach 1:
The system uses automated drift detection algorithms that monitor data distribution changes and trigger re-segmentation only when necessary, eliminating the need for continuous manual updates while maintaining classification accuracy. The proxy model independently identifies when segmentation parameters need adjustment.
Solution Approach 2:
The system dynamically adjusts segmentation parameters based on detected data drift, modifying only the necessary parameters rather than performing complete re-segmentation. This selective parameter adjustment maintains accuracy while reducing computational overhead and operational expenses.
2Measurement precision
If subject matter experts continuously label new data points to maintain segmentation accuracy, then labeling quality is improved, but productivity and time consumption worsen
Solution Approach 1:
A proxy model serves as an intermediary between raw data and SME labeling, pre-processing and filtering data to identify only those points that require expert review. This intermediary layer maintains labeling quality by ensuring SMEs focus on critical cases while dramatically increasing overall throughput.
Solution Approach 2:
Instead of requiring SMEs to label all new data points, the system applies partial action by having experts label only the subset of data points that exhibit significant drift or uncertainty. This selective approach maintains quality for critical cases while improving overall productivity.
3Productivity
If a proxy model is used for automated labeling to increase throughput, then productivity is improved, but measurement precision may deteriorate
Solution Approach 1:
The system implements feedback loops where the proxy model's automated labeling results are continuously evaluated against actual data drift patterns and segmentation performance. This feedback mechanism allows the system to adjust the proxy model's confidence thresholds, ensuring high throughput while maintaining precision through iterative improvement.
Data Source
AI summary
Techniques are provided for segmentation of data points after a dimension reduction. A proxy model is then trained based on results of the segmentation. The proxy model provides low latency high throughput labeling of additional data points, without the need to reduce dimensions of the additional data points. A second segmentation is performed with results of the second segmentation compared to that of the first segmentation. When results of the comparison meet certain criterion, configuration parameters of the segmentation are modified. For example, in some embodiments, a user interface is provided that displays shapley values indicating a mapping from the high dimension data to the segmented data. Input is then received that modifies the configuration parameters.