Automated Anomaly Scenario Generation for ML Model Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The current Managed Detection and Response (MDR) services rely heavily on manual data analysis, making the hunt process slow, tedious, and error-prone. Human analysts struggle to detect subtle patterns in large volumes of data, leading to missed cyberattacks and inefficiencies in data processing.
Innovation Solution
A machine learning anomaly detection system is implemented, which includes a detection pipeline with multiple stages: pre-processing, dimensionality reduction, anomaly scoring using machine learning algorithms, and an interpretability layer. This system uses unsupervised machine learning models that can learn from newly observed data without manual labeling, and it prioritizes alerts to reduce the workload for security analysts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual data analysis is used in MDR services, then human analysts can detect subtle patterns, but the hunt process becomes slow and tedious
Solution Approach 1:
The patent replaces manual mechanical analysis with automated machine learning systems. The anomaly detection system uses unsupervised learning models to automatically identify patterns and anomalies in hunt data, substituting human manual review with algorithmic processing that maintains detection accuracy while dramatically improving processing speed and scalability.
Solution Approach 2:
The patent introduces an intermediary anomaly detection system that sits between raw data collection and human analyst review. This intermediate layer automatically pre-processes, scores, and prioritizes data, filtering out obvious anomalies and presenting only high-value cases to human analysts, thus accelerating the overall hunt process while preserving human expertise for complex cases.
2Measurement precision
If manual labeling is required for training machine learning models, then model accuracy improves, but the system becomes slower and more resource-intensive
Solution Approach 1:
The patent implements self-service through automated label generation. The system uses unsupervised anomaly detection to automatically identify and label anomalous instances from raw data without human intervention. These automatically generated labels are then used to train and retrain machine learning models, creating a self-sustaining system that eliminates the time-consuming manual labeling process while maintaining model accuracy.
Solution Approach 2:
The patent performs preliminary anomaly detection and labeling before formal model training. By pre-identifying anomalous patterns and generating initial labels automatically, the system prepares high-quality training data in advance, significantly reducing the time required for model training and iteration cycles.
3Reliability
If all anomalies are reported to security analysts, then comprehensive detection is achieved, but analysts are flooded with alerts
Solution Approach 1:
The patent applies local quality by implementing anomaly scoring and prioritization that treats different data points differently based on their anomaly severity. Instead of uniform reporting, the system calculates anomaly scores for each instance and selectively reports only those exceeding threshold values, concentrating analyst attention on the most critical cases while maintaining comprehensive detection capabilities.
Solution Approach 2:
The patent extracts and separates high-value anomalies from the general data stream through automated scoring and filtering. By extracting only the most significant anomalies that exceed predetermined thresholds, the system removes low-value noise from the analyst workflow, reducing alert fatigue while preserving detection completeness for critical threats.
Data Source
AI summary
An anomaly detection system is disclosed capable of reporting anomalous processes or hosts in a computer network using machine learning models trained using unsupervised training techniques. In embodiments, the system assigns observed processes to a set of process categories based on the file system path of the program executed by the process. The system extracts a feature vector for each process or host from the observation records and applies the machine learning models to the feature vectors to determine an outlier metric each process or host. The processes or hosts with the highest outlier metrics are reported as detected anomalies to be further examined by security analysts. In embodiments, the machine learnings models may be periodically retrained based on new observation records using unsupervised machine learning techniques. Accordingly, the system allows the models to learn from newly observed data without requiring the new data to be manually labeled by humans.