Cyber-Threat Score Generation Using ML and Source Quality Metrics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cybersecurity systems struggle to accurately evaluate and respond to cyber threats due to the variability in quality of threat intelligence sources, leading to high false positive and false negative rates.
Innovation Solution
A machine learning-based method that quantifies the quality of cybersecurity event data sources using quality metrics such as true positive, false positive, true negative, and false negative rates, and incorporates these metrics into a cyber-threat score generation system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple cybersecurity sources are used to evaluate threats, then the coverage and detection capability are improved, but the false positive and false negative rates increase due to varying source quality
Solution Approach 1:
The patent applies local quality by assigning different quality metrics to different sources based on their individual performance characteristics. Each source receives a quality score reflecting its true positive rate, false positive rate, and other performance metrics, allowing the system to weigh contributions from high-quality sources more heavily while downweighting or filtering low-quality sources. This resolves the contradiction by maintaining comprehensive multi-source coverage while ensuring that only reliable classifications contribute significantly to the final threat assessment.
2Ease of operation
If all source votes are aggregated equally, then the system is simple to operate, but the reliability of the cyber-threat score decreases due to varying source quality
Solution Approach 1:
The patent changes the parameter of vote aggregation from equal weighting to quality-based weighting. Instead of treating all source votes uniformly, the system introduces quality metrics as weighting parameters that modulate the influence of each source's vote. The quality-adjusted aggregation formula incorporates true positive rates, false positive rates, and other performance indicators to dynamically adjust the weight of each source's contribution, thereby maintaining operational simplicity while significantly improving the reliability of the final cyber-threat score.
3Measurement precision
If quality metrics are calculated for each source, then the precision of threat classification is improved, but the complexity of the system increases
Solution Approach 1:
The patent applies self-service by implementing automated quality metric calculation and update mechanisms. The system automatically evaluates source performance based on observed classifications and outcomes, continuously updating quality metrics without requiring manual intervention. This automated self-assessment approach enables precise quality measurement while minimizing the operational complexity burden, as the system performs quality assessments autonomously using predefined algorithms and performance data.
4Measurement precision
If the system filters out low-quality sources, then the false positive rate is reduced, but the coverage of threat detection is reduced
Solution Approach 1:
The patent applies dynamics by implementing adaptive quality-based weighting instead of static filtering. Rather than completely excluding low-quality sources, the system dynamically adjusts their weight based on their quality metrics. High-quality sources receive higher weights, while low-quality sources receive lower weights but remain included in the aggregation. This dynamic approach reduces false positives from poor-quality sources while preserving their contributory role, thereby maintaining broad threat detection coverage without sacrificing precision.
Data Source
AI summary
A cyber-security analysis method uses machine learning (ML) technology to classify cyber-threat indicators, for example, as malicious or benign, by generating a threat score. The method includes receiving, at a compute device, a cyber-threat indicator (IUE) and associated verdicts from a set of sources. Augmenting the verdicts associated with the IUE with verdicts associated with at least one related indicator having a defined relationship with the IUE. The relationship between the IUE and the at least one related indicator can be operational, e g., based on an administrative domain, or functional, e.g., based on a protocol specification. The cyber-threat score is generated for the IUE based on the ML model and the combined verdicts of the IUE and the at least one related indicator.