Network Threat Detection With Real-Time and Batch Anomaly Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing security systems struggle to detect unknown threats and insider attacks, as they often rely on traditional rules-driven methods that are unable to scale or process vast amounts of data effectively, and machine-learning based approaches may fail to identify threats that human analysts can recognize.
Innovation Solution
A hybrid approach combining rules-based analysis with machine-learning techniques to enhance network security systems, allowing for scalable and insightful detection of security threats by integrating machine-learning based anomaly detection models with user-specified rules to identify anomalies and threats.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional rules-driven methods are used for threat detection, then the system can detect known threats with high precision, but the system cannot scale to process vast amounts of data and fails to detect unknown threats
Solution Approach 1:
The patent combines rules-driven analysis with machine-learning based anomaly detection into a hybrid system. The rules engine processes structured data to detect known threats with high precision, while the machine-learning model analyzes unstructured and structured data to detect unknown threats and patterns, together achieving both precision and scalability
2Productivity
If machine-learning based anomaly detection is used, then the system can detect unknown threats and scale to vast data, but the system produces false positives that human analysts can recognize
Solution Approach 1:
The patent introduces a hybrid anomaly score computation mechanism that acts as an intermediary between machine-learning predictions and final threat detection. The system combines machine-learning anomaly scores with rules-driven analysis results, using a hybrid scoring mechanism that reduces false positives while maintaining the ability to detect unknown threats at scale
3Reliability
If a hybrid approach combining rules-based and machine-learning methods is used, then the system can reduce false positives and improve threat detection, but the system complexity increases
Solution Approach 1:
The patent segments the threat detection system into distinct modular components: a rules-driven analysis module for structured data processing, a machine-learning based anomaly detection module for pattern recognition, and a hybrid anomaly score computation module that integrates both approaches. This segmentation allows each component to specialize while maintaining overall system manageability and reliability
Data Source
AI summary
First event data, indicative of a first activity on a computer network and second event data indicative of a second activity on the computer network, is received. A first machine learning anomaly detection model is applied to the first event data, by a real-time analysis engine operated by the threat indicator detection system in real time, to detect first anomaly data. A second machine learning anomaly detection model is applied to the first anomaly data and the second event data, by a batch analysis engine operated by the threat indicator detection system in a batch mode, to detect second anomaly data. A third anomaly is detected using an anomaly detection rule. The threat indictor system processes the first anomaly data, the second anomaly data, and the third anomaly data using a threat indicator model to identify a threat indicator associated with a potential security threat to the computer network.