Anomaly Detection Threshold Setting via ROC Curve Differentiation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional anomaly detection systems face challenges in setting an optimal threshold that balances precision and false positives, leading to laborious efforts and suboptimal performance.
Innovation Solution
A system and method that adaptively estimates and sets an anomaly detection threshold by using a processor to create a ROC curve, estimate the true negative rate based on user-input minimum true positive rate and differentiation slope, and set the threshold accordingly, minimizing precision loss and false positives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a fixed TPR threshold (N%) is set for anomaly detection, then the anomaly detection performance can be measured using ROC curve, but a TPR loss of (100-N)% occurs unconditionally and laborious work to set appropriate N is required
Solution Approach 1:
The system automatically determines the optimal threshold by calculating the differentiation slope from the ROC curve and identifying the point where the first-order differentiation value equals this slope. This self-service mechanism eliminates the need for manual threshold setting while maintaining measurement precision through adaptive threshold selection based on the actual data distribution.
Solution Approach 2:
The invention changes the parameter selection approach from fixed TPR values to a dynamic threshold determined by the differentiation slope of the ROC curve. By identifying the point where the first-order differentiation value equals the differentiation slope, the system adapts the threshold parameter to the specific characteristics of the data, eliminating manual intervention and reducing time loss.
2Ease of operation
If a fixed TPR threshold is used for anomaly detection, then the measurement scheme is simple, but precision loss occurs and appropriate threshold setting requires manual effort
Solution Approach 1:
The system uses the ROC curve and its differentiation properties as feedback to automatically determine the optimal threshold. By calculating the differentiation slope and identifying the point where the first-order differentiation value equals this slope, the system incorporates feedback from the data distribution itself, maintaining ease of operation while eliminating precision loss associated with fixed thresholds.
3Object-generated harmful factors
If the TPR is fixed to minimize false positives, then the false positive rate is reduced, but the true positive rate loss increases and threshold setting becomes complex
Solution Approach 1:
The invention transitions from static fixed TPR thresholds to a dynamic threshold determined by the differentiation slope of the ROC curve. This dynamic approach allows the system to adapt to the specific data characteristics, optimizing the balance between false positives and true positive rate without requiring complex manual setting procedures.
Data Source
AI summary
The present disclosure relates to a system for detecting an anomaly and a method for setting a threshold thereof. The method includes setting a minimum true positive rate (TPR) and a differentiation slope based on a user input, creating a receiver operating characteristic (ROC) curve using anomaly data and normal data, estimating a true negative rate (TNR) from the ROC curve based on the minimum TPR and the differentiation slope, and estimating and setting the threshold based on the estimated TNR.