Security Platform Ground Truth Recalculation for Prediction Model Calibration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Security management platforms face challenges in continuously validating and calibrating prediction models due to a lack of ground truth data, which affects the detection of malicious behavior and security issues in computing arrangements.
Innovation Solution
A system that receives event data and threat intelligence from preceding time intervals to compute and recompute ground truth data, applying prediction models to generate predictions and iteratively adjust them based on comparisons, ensuring continuous validation and accuracy of security models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If ground truth data is used to validate prediction models in security management platforms, then detection accuracy of malicious behavior improves, but the system becomes unreliable when ground truth data is unavailable or outdated
Solution Approach 1:
The system performs preliminary actions by continuously collecting and storing event data, security alerts, and threat intelligence information in advance. This accumulated data serves as a foundation for generating synthetic ground truth data when actual ground truth is unavailable, ensuring validation can proceed without interruption and maintaining reliability while improving detection accuracy.
2Reliability
If prediction models are continuously validated and calibrated, then detection of malicious activities improves, but computational resources and time consumption increase
Solution Approach 1:
The system implements continuous validation and calibration by processing event data and security alerts in real-time streams rather than periodic batches. The synthetic ground truth generation and model calibration occur continuously as new data arrives, maintaining constant detection reliability without requiring separate validation cycles that would consume additional time and resources.
Solution Approach 2:
The system performs self-validation and self-calibration using its own accumulated event data and security alerts to generate synthetic ground truth. This self-service approach eliminates the need for external manual validation processes, reducing time consumption while maintaining continuous detection reliability through automated iterative calibration.
3Reliability
If synthetic ground truth data is generated from event data and security alerts, then availability of validation data improves, but data quality and accuracy may deteriorate
Solution Approach 1:
The system introduces threat intelligence information as an intermediary element that bridges event data and synthetic ground truth generation. This intermediary layer enriches the synthetic ground truth with external threat context and indicators, improving accuracy while maintaining availability. The threat intelligence acts as a mediator that enhances the quality of synthesized data without compromising its availability for continuous validation.
Solution Approach 2:
The synthetic ground truth is constructed as a composite of multiple data sources including event data, security alerts, and threat intelligence information. This composite approach combines the strengths of each data source to create validation data that is both available and accurate, leveraging the complementary nature of different data types to overcome the limitations of any single source.
4Measurement precision
If event data and threat intelligence are continuously collected and processed, then detection capability improves, but system complexity and data processing load increase
Solution Approach 1:
The system segments the complex data processing task into distinct functional modules: event data collection, security alert processing, threat intelligence integration, synthetic ground truth generation, and model validation. This segmentation allows each module to handle specific aspects of data processing independently, reducing overall system complexity while maintaining comprehensive detection capability through the coordinated operation of specialized components.
Data Source
AI summary
Updating ground truth data in a security management platform is disclosed. One example is a system including at least one processor and a memory storing instructions executable by the at least one processor to receive, in a security management platform, event data relating to a plurality of events corresponding to operation of a computing arrangement in a current time interval, and computing ground truth data for the current time interval based on the received event data, and threat intelligence data from time intervals preceding the current time interval. A prediction model is applied to generate predictions for the current time interval based on the received event data. Ground truth data is re-computed for the time intervals preceding the current time interval based on a comparison of the generated predictions and the computed ground truth data.