Dynamic Decision-Making Threshold for Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing solutions for detecting malicious events in communication networks do not consider use-case specific characteristics of different devices, leading to high costs due to manual threshold settings that result in false positives and false negatives, and do not utilize risk and cost values for decision-making.
Innovation Solution
A device and method for dynamically determining a decision-making threshold by obtaining environment threat data, computing anomaly predictions, and adapting the threshold based on target metric scores that include cost values and performance metrics such as precision, recall, and F1 score, to optimize anomaly detection for different devices and network topologies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a pre-defined default threshold is used for anomaly detection, then the system is simple to operate, but it results in high false positive and false negative rates across different device types
Solution Approach 1:
The patent implements dynamic threshold adjustment by continuously learning from labeled threat events and metric scores. The threshold is no longer static but adapts over time based on environmental feedback, allowing the system to maintain high detection accuracy across different device types without manual reconfiguration.
Solution Approach 2:
The system performs self-optimization by automatically adjusting thresholds based on its own performance metrics and labeled data. Through the feedback loop involving metric score computation and threshold adaptation, the system improves its own detection accuracy without external intervention, reducing false positives and false negatives autonomously.
2Measurement precision
If manual threshold setting is performed for different device types, then detection accuracy improves, but operational complexity and costs increase significantly
Solution Approach 1:
The patent creates a universal threshold adaptation mechanism that works across all device types (handheld devices, IoT devices, etc.) through a common learning framework. Instead of requiring separate manual configurations for each device type, the system uses a unified approach that automatically adapts to different device characteristics through metric score computation and feedback-based threshold adjustment.
Solution Approach 2:
The system implements a feedback loop where metric scores are computed based on labeled threat events and current thresholds, then used to adapt and optimize thresholds for future detections. This continuous feedback mechanism eliminates the need for complex manual threshold management while maintaining high detection accuracy across diverse device types.
3Reliability
If a low threshold is set to reduce false negatives, then more threats are detected, but false positive rate increases leading to higher operational costs
Solution Approach 1:
The patent dynamically changes the threshold parameter based on computed metric scores and labeled threat events. Instead of using a fixed low threshold that generates many false positives, the system adjusts the threshold parameter adaptively, finding the optimal balance between detecting all threats and minimizing false positives, thereby reducing operational costs while maintaining detection completeness.
Solution Approach 2:
Through the feedback mechanism, the system learns from the consequences of threshold settings. When false positives occur, the metric scores reflect this, and the threshold adaptation process adjusts future thresholds to reduce such occurrences. This feedback-driven optimization reduces operational costs by minimizing unnecessary responses to false alarms while maintaining reliable threat detection.
4Loss of energy
If a high threshold is set to reduce false positives, then operational costs decrease, but false negative rate increases compromising security
Solution Approach 1:
The system dynamically adjusts the threshold parameter upward or downward based on computed metric scores and security requirements. Instead of using a permanently high threshold that would miss threats, the system adapts the parameter to maintain security reliability while controlling operational costs, ensuring that the threshold is high enough to reduce false positives but low enough to detect actual threats.
Solution Approach 2:
The feedback loop ensures that when false negatives occur (threats missed due to high threshold), the metric scores capture this performance degradation. The threshold adaptation process then adjusts future thresholds to prevent such occurrences, maintaining security reliability while still benefiting from reduced false positives. This feedback mechanism prevents the system from compromising security in pursuit of cost reduction.
Data Source
AI summary
A method for determining a Decision-Making Threshold, DMT, comprising obtaining environment threat data (104) comprising one or more events, each event associated with a label providing an indication of a threat event. The method further comprises obtaining the target metric scores (204); computing anomaly predictions relating to if the events are malicious or not and comparing the anomaly predictions with a DMT to obtain malicious threat predictions. The method further comprises computing current metric scores (303) for the environment threat data using at least the labels and the malicious threat predictions for the environment threat data; comparing the computed current metric scores with the target metric scores and adapting the DMT depending on if the computed metric scores are within a range of the target metric scores or not.