Threat Alert Spike Filtering for Reduced-Bias ML Disposition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning-based threat disposition systems suffer from bias due to incorrect or inconsistent labeling of alerts, exacerbated by alert spikes and bulk actions, which affect predictive accuracy and resource efficiency.
Innovation Solution
A system and method for detecting alert spikes, eliminating duplicate alerts, and revising inconsistent labels using similarity-based grouping and time-weighted ground truth determination to enhance the machine learning model's accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If alert spikes are processed without filtering, then the system responds to all potential threats, but bias is introduced into the machine learning model due to duplicate alerts
Solution Approach 1:
The system performs preliminary actions by detecting alert spikes and removing duplicate alerts before the machine learning model processes the data. This preprocessing step ensures that biased duplicate alerts do not enter the training dataset, thereby improving predictive accuracy without requiring changes to the model architecture itself.
Solution Approach 2:
The patent introduces an intermediary component (the alert spike detection and duplicate removal mechanism) that sits between the alert generation system and the machine learning model. This intermediary filters out harmful duplicate alerts while preserving legitimate threats, acting as a mediator that protects the model from bias without eliminating the need for comprehensive threat monitoring.
2Measurement precision
If all alerts are processed individually, then each alert receives thorough analysis, but resource consumption increases significantly during alert spikes
Solution Approach 1:
The system extracts and removes duplicate alerts from the processing stream, separating harmful redundant data from legitimate threats. By taking out only the necessary unique alerts for further analysis, the system maintains measurement precision for genuine threats while dramatically reducing resource consumption during high-volume alert periods.
Solution Approach 2:
The patent implements a discarding mechanism that eliminates duplicate alerts from further processing. By discarding redundant alerts that provide no additional value, the system recovers significant computational resources while maintaining the ability to thoroughly analyze unique, non-duplicate alerts that may represent actual threats.
3Quantity of substance
If duplicate alerts are retained in the training dataset, then the dataset size increases providing more training examples, but inconsistent labeling occurs reducing model performance
Solution Approach 1:
The system performs preliminary filtering to remove duplicate alerts before they are incorporated into the training dataset. This ensures that only unique, consistently labeled alerts enter the training process, maintaining label consistency without sacrificing dataset quality. The preprocessing step prevents the introduction of inconsistent labels that would otherwise degrade model performance.
Solution Approach 2:
An intermediary filtering mechanism is introduced between alert generation and dataset creation. This intermediary ensures that only non-duplicate alerts with consistent labels are added to the training dataset, acting as a quality gate that maintains manufacturing precision (label consistency) while still building a substantial training set from unique alerts.
Data Source
AI summary
Mitigating bias in a machine learning-augmented threat disposition platform can include generating a group of alerts in response to determining a similarity among the alerts. The alerts are generated in real time by a threat monitoring tool in response to one or more potential threats to a networked computing system. One or more alert spikes can be determined by partitioning the group into one or more alert spike subgroups. Each alert spike subgroup corresponds to an alert spike and contains two or more similar alerts that were generated within a predetermined time interval of one another. Duplicate alerts in each alert spike can be eliminated and each non-discarded alert labeled. The labeled alerts are used for training a reduced-bias machine learning model.