Malware Detection Model With Anomalous Data Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning-based malware detection systems face challenges in maintaining the reliability of decision-making as the representativeness of pre-existing training data deteriorates over time, leading to decreased performance in identifying malicious objects.
Innovation Solution
A method involving the use of two models: a malicious object detection model and an anomalous data detection model, where the latter validates the suitability of incoming objects against the training set, allowing for continuous improvement and updating of the training data by incorporating deviating objects, thereby enhancing the predictive system's precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a machine learning model is trained on pre-existing training data and deployed for malware detection, then the model can identify malicious objects based on learned patterns, but the representativeness of the training data deteriorates over time as threats evolve, leading to decreased detection performance
Solution Approach 1:
The patent implements a feedback mechanism where the system continuously monitors detection results and uses them to identify deviations from the training data distribution. This feedback loop enables automatic detection of when training data representativeness deteriorates, triggering retraining processes to restore performance without manual intervention.
Solution Approach 2:
The system performs self-diagnosis by automatically detecting when incoming data deviates from training data characteristics using statistical tests (e.g., Kolmogorov-Smirnov test). This self-service capability allows the system to identify performance degradation causes and initiate corrective actions without external monitoring, maintaining reliability over time.
2Reliability
If the training set is continuously maintained and the model is periodically re-trained to maintain performance, then detection accuracy can be preserved, but system complexity and computational resources increase
Solution Approach 1:
The patent performs preliminary analysis by characterizing the training data distribution and establishing baseline statistical parameters before deployment. This preliminary action creates a reference framework that enables efficient anomaly detection during operation, avoiding the need for complex continuous retraining by preparing detection criteria in advance.
Solution Approach 2:
The system monitors changes in data distribution parameters (e.g., mean, variance, entropy) to detect when training data representativeness deteriorates. By tracking parameter changes rather than performing full model retraining, the system maintains detection accuracy while reducing computational complexity and resource requirements.
3Measurement precision
If two detection models are used (malicious object detection model and anomalous data detection model), then the precision of malware identification is enhanced by distinguishing classification errors from dataset issues, but the device complexity increases
Solution Approach 1:
The patent segments the detection task into two distinct functions: a malicious object detection model for identifying malware and an anomalous data detection model for detecting training data representativeness issues. This segmentation allows each model to specialize in its specific function, improving overall detection precision while enabling independent optimization and maintenance of each component.
Solution Approach 2:
The anomalous data detection model serves as an intermediary between the training data and the malicious object detection model. It monitors whether incoming data conform to training data characteristics and flags deviations, allowing the system to distinguish between actual malware detections and false positives caused by data distribution shifts, thereby enhancing precision.
Data Source
AI summary
There are provided measures for machine learning based malware detection systems. Such measures exemplarily include analyzing a set of training data, said set of training data comprising a plurality of training data elements, wherein each of said plurality of training data elements is associated with a respective one of at least two maliciousness related properties, learning a malicious object detection model on the basis of first feature combinations of said plurality of training data elements, said first feature combinations characterizing each of said at least two maliciousness related properties, learning an anomalous data detection model on the basis of second feature combinations of said plurality of training data elements, said second feature combinations characterizing said set of training data, said anomalous data detection model being associated with said malicious object detection model, and providing said malicious object detection model and said anomalous data detection model.